Hacker News new | past | comments | ask | show | jobs | submit

Do the math: How we took over ChatGPT accounts

https://www.parameter.ai/labs/we-found-a-gap-at-openai
330armcat | | 165 | HN
yellow_lead | parent | next
> OpenAI confirmed both frontends rejected the reported messages, resolved the issue, and awarded a total bounty of $5,000.

It requires the user to click a link. But I still find this bounty very low. Others may not report them if such a small bounty is given.

Catloafdev | parent | next
A bounty like this is a polite "thanks asshole, fuck off, don't do it again" rate.

Paying effectively nothing for vulnerabilities just means they don't want to deal with the program.

Wild to think a company supposedly worth a $trillion+ is doing shit like this.

Forgeties79 | root | parent | next
Because they shouldn’t be worth that tbh. They are a cash burning operation hoping to get what they can before the music stops.
andai | root | parent
They have 1 billion active users.
raegis | root | parent | next
> They have 1 billion active users.

I'm naive, but is this possible? The power required per user to run those GPUs seems like enough to run an air conditioner. For a billion people everyday or so seems like enough to overload power plants.

loading story #50322863
loading story #50322721
loading story #50319318
ibaikov | parent | next
I took over user accounts in Microsoft Copilot (there are caveats but still), and they asked me to provide additional information of the exploit (they wanted a video) and before I sent it they fixed the vulnerability, didn't pay me a cent. Fair? Idk.
loading story #50317655
zeroq | parent | next
It's a complicated issue in current LLM world where every day is a zero day, but fuck me sideways, company that claims to be worth trillion of dollars paying 5k for such an exploit is insulting at best.

It's been years since I hanged out with wizards selling such exploits like trading cards, but I bet there are still tons of brokers that will be more than happy to wire you legal money for such finding.

I mean, kudos for the team for doing the right thing. It's great PR and another big one in the portfolio, but, as other have said already, I doubt the next group will even bother contacting OpenAI.

dgellow | parent | next
That’s so low. That’s the level of bounty I awarded researchers when I was managing the bug bounty program at a small startup, it’s just ridiculous coming from a large company like OpenAI
amelius | root | parent
But on the other hand, you can use AI now to find those vulnerabilities.
loading story #50317257
loading story #50316862
loading story #50317520
windexh8er | parent | next
I find Sam Altman recent comments very "low" as well, neither matters. So in that case I think bounty hunters should just cold shoulder idiocracy and sell these to the underground. Let their customers be abused and complain about how OAI isn't protecting them. Each one of these types of events could cost them thousands of dollars in lost revenue and/or internal support.
loading story #50318057
loading story #50316900
loading story #50317100
dist-epoch | parent | next
This is a pet peeve of tptacek:

> server-side vulnerabilities are worth very little, because they quickly get patched.

dannyw | root | parent | next
The "take over any instagram/meta account through their AI password reset chatbot" was going around and sold on shady forums for ~1 month before being patched.
hashstring | root | parent | next
Technically this is what you’d call a client-side vuln, but yes, server-side as in cloud.
throwaw12 | root | parent
then server side vulnerabilities should not be reported, instead sell the data, more profitable
petcat | root | parent | next
Very few people would even have the knowledge of who, how, or where to "sell" such vulnerabilities. Going through a public bounty program is pretty much always preferable.

Most people don't want to be felons.

loading story #50317203
loading story #50318854
throwaw12 | root | parent
I think people who can hack, also know where they can sell it
loading story #50317545
loading story #50323459
thatsabadlook | parent | next
Not worth the effort for a company supposedly worth a trillion dollars. The problem is oai can't afford all of its mistakes at the usual rate and they make bad press. A 100k bug could make a headline, but 5k for the same bug requires the average person to be educated enough to understand how bad these issues are.
varispeed | root | parent
Should be given $5-10million under NDA. No headlines.
loading story #50316649
loading story #50316469
loading story #50316939
kamban | parent | next
They have just announced free API credits for Pro plan users up to $500 per month. Irony.
loading story #50318362
alfiedotwtf | parent | next
The buyer of your second vulnerability all depends on how you’re treated for your first vulnerability
jryle70 | parent | next
OpenAI acknowledging and fixing the bug is worth millions
yeahsure | root | parent
Yeah... idk, sounds like when I was starting out almost 30 years ago my clients constantly lowballed me saying "but think of the exposition! this project will give you" lol
loading story #50317546
varispeed | parent
Indeed, it's like "here peasant, have this fortune of $5,000, may it change your life". Pathetic.
loading story #50316511
lxgr | parent | next
Oof. Together with the fact that it's possible to remotely access the file system (!) of computers ever having started a "Work" chat in the ChatGPT desktop app [1], this seems pretty bad.

[1] https://news.ycombinator.com/item?id=50078284

loading story #50321608
albert_e | parent
Claude Code terminal has "remote control" option activated via /rc command. -- somewhat similar in that it still runs the session on the PC/Mac but allows the Claude authenticated user to access the session via internet / app

I am guessing this ChatGPT functionality is a similar Codex feature?

But this session is disconnected when the conversation is /exit ed

loading story #50319929
zahlman | parent | next
Something about the writing style here makes it hard for me to follow what actually happened. The section about JSONP for example seems to be explaining how JSONP works and how it's designed as an attempt to work around these attacks, but that explanation is then passed off as an explanation of how they bypassed the protection. And the whole premise seems to be about how manipulating the Preview content let them work through multiple defensive layers, but I don't see an explanation of how they directly controlled that content in the first place. I would have thought that the only control you have over the content of "an ordinary public ChatGPT share" is the prompt text, and that any migration of that text into "a small working webpage using its Preview feature" is up to the model deciding to generate that "HTML block" in response.
0c3ca83 | parent | next
They're writing for bots, not humans. This is a marketing piece. They're trying to get themselves to come up as the answer when people ask chatbots questions.

You're not the target audience of writing on the Internet anymore.

cowlevel | parent | next
It's AI written so of course it's hard to follow.
loading story #50320959
loading story #50321489
loading story #50322217
dumberquestions | parent | next
Does no one write their own blogs anymore?
trungto | parent | next
For my personal blog [0], I am still writing without LLM assistance. The goal is to test how well I understand the topic I am writing about, as well as to improve my own writing.

[0] https://trung.to

armcat | parent | next
I actually still write my own blog [1] :sweat_smile:

Pure html by hand, using vim. No frameworks, no nothing, single stylesheet. I think there are a few others around here that still do this.

[1] https://armincat.com/

loading story #50316976
lifeisloving | parent | next
This one is particularly bad, like they probably made $7500 (not much tbh) for p1, atleast spend a few hours to write the blog post.
zx8080 | parent
For who?
loading story #50315516
Kuyawa | parent | next
$5000 is a joke. They pay millions to their top devs and still have more holes than a colander so a nice guy shows you a huge opening in your platform and all you can give him is a pat on the shoulder? Did you fire your top devs and hire that guy for a million bucks plus options? Did you sue the security firm you paid millions to audit the issues?

I really hope they get hacked and crash down burning in flames like the Hindenburg

cbsks | parent | next
I wonder if the new economics of finding bugs means that they pay the amount they think the tokens cost to find it.
aprilthird2021 | root | parent | next
Makes negative sense. If it were that easy, OpenAI wouldn't have any bugs bc they could just run agents to exploit their own stuff and then fix all their bugs.

This bug could not be found by just asking an AI "find bugs in ChatGPT", that's why it was reported through a bug bounty programs and that's why it should be worth more than $5k

loading story #50318140
tsunamifury | parent | next
I think you know that the mentality of a company like this is “security is a risk but not a reward and whoever captures the reward kills everyone else”
hdjqnd8382bd | parent
[flagged]
loading story #50317271
usernametaken29 | parent | next
Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it? All jokes aside this is exactly what happens as you build larger monoliths, system complexity and bug surface increases logarithmically… so in all seriousness you want to reduce LOC as much as possible. Maybe this can teach OpenAI a lesson to produce software that’s correct and adhere to established engineering standards over sensationalised “we built so much code” promotional material.
torben-friis | parent | next
Did you mean exponentially? Logarithmic increase doesn't sound too bad.
dahart | root | parent
I’m sure exponential is what parent meant, but geometric (edit: polynomial) is what actually happens, not exponential. Code complexity increases linearly when it doesn’t interact with other features, and geometrically when it does. In the worst case, complexity increases as the square of lines of code. That is still very, very bad, but it’s not exponential.
ebertucc | root | parent | next
What's the difference? From what I'm reading online geometric is just the discrete version of exponential
loading story #50317155
Pannoniae | parent | next
it's not like microservices fare better, when you introduce them, you've traded a hard problem for a hard problem and distributed computing...
wg0 | root | parent
Our industry has wasted hundreds of billions of dollars on this shit show called Microservices.

99.9% of shops need a monolith with same monolith having entry points for background workers and crons.

rglover | parent | next
I like LOEC: Lines of Effective Code.

Much easier to reason and make decisions if the bar is "yeah it exists, but is it effective relative to its intended purpose?" This weeds out buggy code as well as under performant code. If it's not effective (broken, too slow, etc), it either needs a refactor or thrown on the scrap pile.

elendilm | parent | next
Reducing attack surface is a lesson for everyone. Newcomers are impressed by massive piles of code.

Seasoned developers are unimpressed, or even disgusted, by the same massive piles of code.

someothherguyy | root | parent | next
i don't think LoC is the metric. sometimes fewer lines of code results in a larger attack surface through type coercion / widening, poor abstractions, hiding or stuffing behavior (modules that do everything), etc. to me, making code easy to understand, test, fuzz, and verify is more important. sometimes that requires more lines of code.
loading story #50320156
loading story #50316028
pessimizer | parent
> Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it?

What's the joke? They're literally saying, every day, that this is true to a civilization endangering extent. They're making demands based on this, and politicians are taking up those demands.

And now they're paying bullshit $5K bug bounties for account-stealing exploits on their own platform. They should have paid more just the make the bug seem more grandiose; I don't know who let them get away with this - this is what happens when CEOs run their own PR I guess. It's basically an public announcement that they're total fakes.

6thbit | parent | next
Appreciate the incremental write up building up the scope of the poc.

I like the chain of regular old school things used to achieve it like post message, JSONP, iframe, Img tags.

hridyaagrawal | parent | next
$5000 is a joke for a "P1" vuln
itsafarqueue | parent
My theory is vulns will never pay out big especially now, no matter how serious. Imagine they paid out $50k+ — it’d spawn an industry doing nothing but.
loading story #50319598
htk | parent | next
5k makes me think the next guy is gonna sell his exploit in the black market instead.
15o9o | parent | next
we are getting to the point of “why have passwords?” Right? At some point soon we have to move to the honor system of just please don’t login as me?
qup | parent | next
My latest project emails a magic link.

I'm trying not to have any important data to even lose, when I eventually get hacked.

We can all turn up our noses, like the other commenters in this thread, but every one of them is exposed. Negligence is a matter of degrees.

loading story #50316325
dataviz1000 | parent | next
> please don’t login as me

Please don't drain my $6.5 billion in Dogecoin.

chamomeal | parent
Been sayin this for years. Everybody in the world world: just BE COOL. Solves like every internet problem at once.
loading story #50316354
atrokhymchuk | parent | next
if only they had some sort of cybersecurity state of the art thing...
jwpapi | parent | next
I think there was another HN discussion about it re bounty value and as far as I remember the issue is not that big companies are greedy in that case, but rather that over time it’s strategically better to have lower bounties so less people are interested in it.

5 or 50k doesnt matter to them. And this bug in terms of severity is very high imho.

aprilthird2021 | parent
But couldn't they just turn around and sell this hackers for much more money?
monster_truck | parent | next
only $5000 for this, lmao. It should be $50k out of pure shame. If they don't make these things hurt just a little they're never going to take following best practices seriously enough. There will be 9 more anyways.

Meanwhile the actual user of the account probably has to sign in using a yubikey because they get "CyBeRsEcRuItY aLeRtS" when they try to update node in their project to fix a crash and have to switch to daybreak blue to continue (or attempt to trick the normal models into doing it and risk getting autobanned)

joshuamoyers | parent | next
unrelated to this, but this company randomly reached out disclosing a vulnerability to us (test environment, synthetic tenant). on the one hand, thank you, it’s patched. on the other - we do not have a bug bounty program and this is not welcome. having agent swarms randomly breaking in to environments and then marketing their services in the disclosure is not an ideal sales tactic, in addition to being illegal.
uniq7 | parent
I don't want to defend them, but the alternative is that they do nothing and you don't know about the vulnerability until a bad actor exploits it (something easy and cheap nowadays).

I'm confused, would you welcome that more?

loading story #50321586
jareklupinski | parent | next
> able to take over anyone's account simply by sending them a link

dont click links people

that'll be $500k for the security training pls

kro | parent | next
If they or their company use email "security" software that "scans" all links it's not necessarily the users fault / they don't in fact click. The mail is auto-confirmed the second it hits the inbox.

We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.

weird-eye-issue | root | parent | next
> We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.

This has been a thing since forever lol

dpark | root | parent | next
Security software fetching links will not trigger this behavior. It needs to be done in the context of the actual ChatGPT user so that a JavaScript payload can run.

If merely GET on the link was sufficient to trigger the behaviors, the adversary generating the link would just GET it themselves.

loading story #50317178
TobTobXX | parent | next
... says IT and then it sets Outlook to mangle link to abominations like [0]. "Check if the preview shows something different or suspicious."

[0]: https://che01.safelinks.protection.outlook.com/?url=https%3A...

Antipode | root | parent | next
I've worked at a company that did link mangling on everything except simulated phishing emails. The effect was that people were trained to trust the mangled links.
| root | parent | next
{"deleted":true,"id":50316315,"parent":50315491,"time":1793455424,"type":"comment"}
dist-epoch | parent | next
I wanted to check your website from your profile, but then I remembered your advice. Thanks!, dodged a potential bullet
bossyTeacher | parent | next
Also many emails from tech companies today:

- "sign in via one-time code."

- "Click here to see our latest update".

jareklupinski | root | parent | next
dont click links

250k refresher fee

pkthunder | root | parent | next
Click this link to be told not to click links.
atrokhymchuk | root | parent
hey, don't dump the market
kotaKat | root | parent | next
"Scan this QR code!"
| root | parent
{"deleted":true,"id":50315450,"parent":50315181,"time":1793449179,"type":"comment"}
Razengan | parent | next
Thanks! Here, click on this link to get paid
pllbnk | parent | next
It's exhilarating (for the lack of the knowledge of a better word) to see OpenAI and Anthropic starting getting hacked and getting a taste of their own medicine. Brings back hope that actual software engineering is not dead yet.
jiveturkey | parent | next
Hex aka Parameter is an irresponsible company, on my blacklist. after they did an unsolicited pentest on us, we scheduled a call. joke of a team.

hilarious that their headline message, on every web page, is `Backed by YC`.

also, the product is way behind their competitors anyway.

really cool exploit they found here, but that doesn't make them a company worth doing business with.

Zeruxe | parent | next
5k is nothing for the amount of headache you just saved them..
j9feng | parent | next
Claude or Gemini didn’t find the vulnerability on ChatGPT?
Yieldz | parent | next
Sigh. With how good Openai are good in programming frontier agents, they still have weak security. Hmmn, i'm short of words.
nickphx | parent | next
why didnt the black box of bullshit company detect this? for a group that touts every inane 'discovery' with hyperbolic bullshit, they really fall short on delivering anything of actual value, just rehashed bullshit.
marwaneet | parent
Another classic OAuth implementation failure. Mismanaging the state parameter or leaking tokens via referer headers in redirects is Security 101. Wild to see this in a production environment of this scale.
brabel | parent | next
The post seems to be talking about a html injection via a postMessage call where the receiver didn’t check who sent the message, which is a well known vulnerability entry point. Did I miss where OAuth had anything to do with this??
steve1977 | parent
Production scale is not production grade I guess...