If you're on 2.1.317 or later, you'll have this check included. You can turn it off via /plugins.
Best practice will forever be to enforce your guard rails in addition to Anthropics guardrails.
Offshoring understanding is the trend that many companies will pay in the form of comprehension debt.
Somebody on Reddit figured out they just forgot to add a single line permission request into the app. It's been broken now for months.
I just find this wildly hilarious given the 'world ending all powerful AGI is here' rhetoric they're running with at the same time.
Maybe before worrying about permanent 15% GDP growth or human annihilation, what if we started by putting an agent on user issues and solving them pro-actively?
I know the kool aid is tasty, but we might be getting a bit ahead of ourselves here guys. You can fantasize about human annihilation after you figure out how to make a desktop app that doesn't have obvious bugs, I promise :)
Guarding PII is an extra, I guess, and if you want to use a giant cloud-hosted NN to control software on your own device, there are very few default restrictions. And no matter what restrictions there are, when they go beyond prompting, they would have their own ShowHN.
I never give a provider my actual email address, I recommend others do the same. Otherwise, it’s just a matter of time until it is leaked or otherwise part of a data breach.
“lethal trifecta”
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
(a component of my cybersecurity work is red teaming inference, agents, and mcp interfaces)
This is why alignment is important and highly non-trivial
FYI, it's generally bad form to "haha" your way through privacy violations.
Leaking private information is a very big deal for a lot of people.
I disagree, we can likely never fully (or should?) trust models to do these things deterministically, that's why we have harnesses around the agent with built-in constraints.
It is actually in the name: https://en.wikipedia.org/wiki/Harness, https://en.wikipedia.org/wiki/Safety_harness
This is an official Anthropic response???
Anthropic should have been aware of this since the user's login email address was inappropriately added to the system prompt. It's been informed of it by third parties since at least July: https://github.com/anthropics/claude-code/issues/81138
It makes no sense for an email address only used for authentication to be given to Claude Code, so the fix should be to remove it from the system prompt, not to add a plugin.
Which is great! It's good for the community when people give quick updates here, and to make sure that it continues, it's important that we respond respectfully. (I don't think your post was disrespectful, but commenters have crossed that line in other situations in the past.)
If you’re in the EU, send a complaint to your data protection authority.
I imagine quite a few would be very interested in following up and investigating here, and getting Anthropic to drop the email from the system prompt.
Nice that this time we get the company to admit it in public, and that they've been aware of that earlier but only acted now.
I suspect the privacy plugin method is a more sane fix -- ideally, plain old code should be inspecting any web request my agent makes and let me decide if that outbound request should contain certain sensitive strings. I'd also want to put a few bytes of my SSH private key and of any other sensitive keys in the 'do not leak' watchlist as well.
(No, I don't want to make a real "App" to sell on a "Marketplace," or be involved with oAuth, or any other tenant. I just want an API key for my own tenant, that isn't specifically tied to me, so every automation I build won't break when I quit someday.)
Should be this: https://support.atlassian.com/user-management/docs/understan... https://support.atlassian.com/user-management/docs/manage-ap...
the GitHub REST api proxy in Claude Code for the Web mangles HTML in PR descriptions. If you have some third party service (e.g. cubic code review) which adds HTML to the end of your PR description, then ask Claude Code for Web to edit it, the HTML gets turned into entity escapes (< into <, etc). Complain about it to Claude, Claude says it can’t help itself because the proxy in the sandbox is what is doing it, not the actual LLM.
Also from a design perspective, I feel like including email and other personal data into the context is a bad idea for "data minimisation" reasons
The fine for breaching GDPR can be either €20m or 4% of global revenues (whichever is the larger figure), so if I were them I'd do everything possible to sort this issue out ASAP.
Anthropic seems very proud of how many tokens they can use on things like developing Claude Code. Yet somehow it started sending user’s email addresses in the user agent header. And the fix is to deploy a mod? That detects the completely bonkers behavior and suppresses it?
Is the actual quality of the software inversely related to the number of tokens spent? Does any human know what Claude Code does or is supposed to do? Does even Claude plus all its Slack-conversation inputs know?
Today I watched Astra correctly detect a bug (probably untriggerable in practice) in some Important Production Code. And I watched it fail to root cause it, “fix” it by bolting on more complexity, agree (on further questioning) that it still wasn’t fixed, and design an even more complex scheme. With my Mark I human brain, I observed that the actual issue was that a helper function didn’t do what it said on the tin when called in an unexpected context… and fixed the helper directly. Problem solved. Ugly AI-generated extra complex workaround layers avoided. I suppose if I multiplied my token spend by 1e5 and did not increase brainpower at all I could make a truly spectacular mess.
The actual Claude model is a truly spectacular piece of software. Claude Code is a nifty little command dispatching engine. There are very comprehensible third party tools that do basically the same thing, in not many lines of code, where the primary advantage of Claude Code is that Anthropic permits subscription use. (Interestingly OpenAI has no such constraint, and they publish almost the full client sources.)
How many PRs/equivalent per day do you think happen at Google/Microsoft/Meta/etc prior to AI?
Is that not precisely the kind of thing they claim their tools enable?
Literally unlimited tokens on tap and yet they can only manage to make privacy violating slopware
I was part of a discussion on Threads on 25th October where Boris Cherney was tagged multiple times.
And the issue wasn’t even new at that point in time.
[1] https://blog.xkeeper.net/the-cutting-room-floor/self-hosting...
[2] https://github.com/TecharoHQ/anubis/pull/1895/changes/de2312...