Hacker News new | past | comments | ask | show | jobs | submit
hypfer | on: AI agent runs amok in Fedora and elsewhere
> while it started to look off after a while, all the replies were still like this - a bit weird, but still plausible

I believe that we will be seeing the death of "assume good faith", which is not a bad thing, given that this was an exploit vector that has been actively abused for many years now.

"Assume bad faith and work backwards from that, rule out any possible exploits and only then clear the input for processing" will be the new normal.

Which is good. We need friction. Friction makes stuff slow down and work at the speed of humans.

account42 | parent
It is a bad thing. The good response to bad actors abusing good faith is to make sure there are consequences that disincentivize that behavior in the future. Sliding further towards a low trust society means the bad actors winning in the same way that terrorists win when we subject everyone to restrictions as a result.
hypfer | root | parent
You don't slide into a low trust society though.

Quite the opposite. You just add a Wall with a Gate. Inside those walls, you suddenly have a high trust society again.

The issue that is currently breaking reality was that we thought that everywhere could be a "high trust" space. This was proven countless times to be wrong.

Tearing down all walls - as it happened with the assault on friction (thanks hyperscaling) - did not lead to the "high trust" spilling out, but the "low trust" spilling in, essentially.

account42 | root | parent
It's a question where you build that wall. If you build it around the home of your immediate family and keep almost everyone else out then you can hardly be said to have a high trust society. The goal should be to put only those bad actors behind a wall, preferably a physical one.
m4rtink | root | parent | next
Yeah, gated communities like that are usually a clear sign that something bad is happening with the given society - or in a minor cases with the community, if it needs to gate itself from a society that is not failing.
hypfer | root | parent
Sure, but that's a completely different discussion.

Plus that even with such a small scale of the "inside", the thing fails gracefully. It is arguably a failure mode, yes, but it is one that leaves a functioning system (albeit one that stays below its potential).

This is not true for the inversion of the scenario. That does _not_ fail safe but just leaves rubble behind.