I think it's partially for responsibility reasons. Your employee approved the bash call? not our fault then!
Or in one of my favorite coinages: "Moral Crumple Zones".
"Just as the crumple zone in a car is designed to absorb the force of impact in a crash, the human in a highly complex and automated system may become simply a component—accidentally or intentionally—that bears the brunt of the moral and legal responsibilities when the overall system malfunctions."
See https://ferd.ca/notes/paper-moral-crumple-zones.html
Related: "Accountability Sink", a system which is designed to channel responsibility away from the people making the decisions, e.g. a powerless "complaints department".
Yep and the car wasn't self-driving at the moment it crashed.
...because the self-driving feature turned itself off after detecting the crash in the fractions of a millisecond before the crash was recorded
In a rare bit of still-sane news, the US National Highway and Safety Administration staff aren't dumb: Their policy is to consider whether any automation was active in 30 seconds before the crash.
https://www.nhtsa.gov/laws-regulations/standing-general-orde...
loading story #49202115
loading story #49201652
"Uh oh, this is a problem. Welp, I'm outta here, good luck."
If a company advertised an LLM as perfectly safe and then it caused some damage, there would be a case against the company.
Have you used LLM tooling? It comes with warnings and explains that the user accepts the risk. Different levels of warning are supplied for the different levels of autonomy you can enable. The user has to understand the risk as they enable it.
This is not a new concept and it’s not an idea the LLM companies invented. It shouldn’t be surprising to anyone.
It's not new to LLM companies, but it does seem like they're leaning away from the standard "Hey, don't blame us if this doesn't work" and are moving into "It's now your fault any time our software fucks something up" which feels new.
> This is not a new concept and it’s not an idea the LLM companies invented.
I don't recall any prior computer software working so badly that it needed a disclaimer like "Claude is AI and can make mistakes" on its front page. Let alone one so costly.
You must not use any open source software, because much of it comes with a whole paragraph of ”IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY“
That would be fine if Claude was open source and/or unpaid
If you read the fine print of most commercial software you'll see some language to a similar effect. When's the last time Microsoft got sued due to an outage caused by a bug in Windows?
That's actually meaningless in Europe under the Cyber Resiliency Act by the way. They are liable even if they say they aren't.
This sentence only exists to offset law (or case law) have gave damage rights to anyone.
Every piece of software comes with giant disclaimers like this.
You’ve just been clicking past them.
This really isn’t new.