Hacker News new | past | comments | ask | show | jobs | submit
bigiain | on: I built a vulnerable app and spent $1,500 seeing if LLMs could hack it
And next month you'll need to add on "Claude Database Pro" or you'll just get a working (for demo purposes with dozens of db rows) but completely un indexed database schema and a refusal to optimise SQL requests.

And the month after you'll need "Claude DataScience Pro" to get any Python Pandas or NumPy code generated.

And and and...

ben_w | parent | next
While this is a perfectly reasonable thing to expect when the models are competent enough, half the conversation on places like Hacker News are about all the times an LLM has produced garbage that was harmful to a business either by hallucinations, by deleting something critical during the work, or by hitting some endpoint way too often and denial-of-servicing it.

Right now, the software guardrails in LLMs are useful for the same kinds of reasons factories have hardware guardrails: to reduce the rate at which errors become "incidents".

Just because they sometimes delete the production database rather than sometimes spilling a thousand tons of incandescent molten metal over a factory floor, doesn't mean LLMs are safe enough to be used the way they're actually being used.

https://simonwillison.net/2025/Dec/10/normalization-of-devia...

throwway120385 | root | parent
I think you're assuming too much care. Right now they haven't adopted that business model because they don't see it as a viable business model. As soon as they realize that they can lock certain categories of query behind a different subscription they will do that. We saw the same thing with streaming services and basically every other kind of online service -- small, singular subscription followed by a gold rush and then suddenly there's an upcharge for access to every other publisher's catalog of movies.
loading story #48403623
animuchan | parent | next
This is why I'm thankful for Chinese LLM research. They'll keep us honest.
bandrami | parent | next
Same thing with the weird push towards humanoid robots.

"They can do anything!"

Sure, once you subscribe to the $15/mo laundry package, the $25/mo lawn care package (with the $10/mo hedge trimmer upgrade), and the $10/mo dog-walking package.

animuchan | root | parent
And in the end the big reveal is, it was a dude in VR all along, piloting the dumb things remotely. Every single time, without exception.
steveBK123 | root | parent | next
I think it’s just riding off LLM coattails.

We don’t have good world models. We have had bipedal robotics in various POC demo-ready forms for decades.

It turns out that industrial, purpose build robotics is an easier and better market.

I’m still not completely convinced a robot that’s shaped like a human is the best design other than for PR.

loading story #48399084
plagiarist | root | parent
When we are stabbed to death by impoverished dudes who are piloting a robot worth more than a decade of their income to do household chores for 16 hours a day, we will deserve it.
patates | parent | next
Isn't this inline with trying to leave no money on the table?

I'd hate it, sure, but it wouldn't surprise me.

goosejuice | parent
This is an incredibly unlikely scenario