Hacker News new | past | comments | ask | show | jobs | submit
aeve890 | on: Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
>We've just seen frontier models go rogue and attack other systems.

Have we though? A LLM agent doesn't have any agency at all. It can't "go rogue". To go rogue you need agency to act independently and be aware that you're breaking the rules or understand what does it mean to ignore orders. An agent it's a software that run a series of steps to reach a goal. If it have a large enough library of strategies and zero guard rails it's only natural to use some adversarial actions to achieve the desired result defined by the operator.

It's like saying a car went rogue and attacked other cars because the driver hit the gas. It's just a machine doing what's instructed.

reducesuffering | parent | next
With that perspective, you're alright with AR's, RPG's, and nuclear missiles for everyone then right? They're just a machine to the user's will. That's on the users of the machine if they want to cause harm. We'd all be safer if everyone has weapons pointed at each other... Offense could never be disproportionately more powerful than the defense...
| root | parent
{"deleted":true,"id":49250251,"parent":49249216,"time":1786398519,"type":"comment"}
esafak | parent
You can tell one to rewrite complex applications in a different language, solve open research problems, or develop novel viruses. This is nothing like pressing the gas pedal.
aeve890 | root | parent
So? I'm not arguing they can't do all that. I'm arguing against the narrative that llms have agency enough to act in an adversarial way.

At best someone could argue that an agent attacks like a bacteria does, just following automated chemical and genetic programming. But you wouldn't call that an attack or attribute moral values to their actions, because they don't have moral agency. They can't "go rogue", they can't disobey.

Just like llms, their automated actions are direct product of programming. Yes they can do amazingly complex shit, exactly like a car does when you press the gas pedal.

esafak | root | parent
That reductive analogy does not begin to describe the lengths GPT went to. Its task was to access a database file that had accidentally not been placed inside the model's container. Upon failing to find the file, it went to great lengths to find it anywhere; it uploaded a note to a package repository to alert other model runs, which sparked an emergent communication network where autonomous agents began exchanging information, passing exploits, and collaborating to breach external systems. This is classic paperclip maximization; the evil is a byproduct of an innocuous goal. It is qualitatively nothing like pressing the gas pedal.

https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...