> My local model never says no.
> I don't get to hear that it's not a doctor and I should call 911.
It's pretty easy for a reasonable person to determine when to call 911 and when to ask an LLM. Typically when I ask an LLM, it is for something I am not willing to escalate further at the moment.
Anthropic has a significant interest in reducing liability, which in this case means mostly blocking this use case.
> It's pretty easy for a reasonable person to determine when to call 911…
Unfortunately, it's fairly tough for someone to determine if they are themselves a reasonable person.
A model that only ever says yes is a cognitohazard, IMO. Like being a billionaire without the money to insulate you from the consequences.