Why was this the default behavior in the first place?
I'd guess it's simply because it tried to use the context to achieve the goal.
Guarding PII is an extra, I guess, and if you want to use a giant cloud-hosted NN to control software on your own device, there are very few default restrictions. And no matter what restrictions there are, when they go beyond prompting, they would have their own ShowHN.
loading story #50386272
Always use a hidden email service, you cannot trust the LLM or inference provider to secure the email address in this context. If they don’t have the email address, they can’t leak it.
I never give a provider my actual email address, I recommend others do the same. Otherwise, it’s just a matter of time until it is leaked or otherwise part of a data breach.
If you let the LLM see git commands, they have whatever email you use to sign commits.
I use a masked email for Git too. Always have. Last 5 years or so it's just been the GitHub provided mask e.g. <numeric-ID>+<username>@users.noreply.github.com
loading story #50387995
I am a security person, I don’t let the LLM see anything except that which I wouldn’t mind printed on the front page of the New York Times. I treat my LLM context as I would a potentially malicious threat actor. Nothing valuable, sensitive, or personal goes into the context.
“lethal trifecta”
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
(a component of my cybersecurity work is red teaming inference, agents, and mcp interfaces)
{"deleted":true,"id":50386793,"parent":50386046,"time":1793926490,"type":"comment"}
Bingo. Auto mode doesn't mean send private information outbound. There are other deterministic hard points - why not this?
Some sites like Wikipedia have a policy where scripted clients identify themselves, and there's a standard in HTTP to have a contact email in the User-Agent for such cases. Claude apparently tends to decide that this policy is worth following, and we now disallow this programmatically.
This is why alignment is important and highly non-trivial
loading story #50386226
loading story #50386249
loading story #50386568
loading story #50386525
loading story #50389004
Because they didn't think about it or any of the other million things that might go wrong - that would take careful planning.
Because Claude decided so. You think humans write or even review what they ship to run on your computer?
{"deleted":true,"id":50386148,"parent":50386046,"time":1793921849,"type":"comment"}
you'd need to ask one of the billion claude sessions they have doing whatever in their cloud rn