Hacker News new | past | comments | ask | show | jobs | submit
ASalazarMX | on: Tracking Anthropic-discovered vulnerabilities
I'd expect the remaining 80% are delayed not because there's something statistically special about them, but from the massive workload thrown at humans in the loop.

Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.

loeg | parent | next
> Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.

Of course it's trivial to have an LLM generate a fix, too. They probably do. It doesn't mean a project (1) wants to take an LLM-generated fix at all, or (2) can immediately commit an LLM-generated fix without review (which takes time).

MBCook | root | parent | next
Maybe not. And that’s totally fine.

But right now they’re not even offering one. They’re just putting the burden on the project without even trying to help fix it.

They have all the resources in the world, and they won’t even try.

And that’s of course assuming it’s a real bug, as we’ve seen a ton of them don’t seem to be. The last time we went through this, curl reported not only that a bunch of them weren’t bugs, but some of the bugs were listed previously on their mailing list, and that’s probably where the LLM got them.

So not only is it finding small bugs. And nonbugs. It’s reporting previously found bugs.

Pilling work on maintainers. Over and over again. To help? No. To make big numbers to show off for their IPO documents.

loading story #50449814
| root | parent
{"deleted":true,"id":50448386,"parent":50448231,"time":1794357780,"type":"comment"}
ameliaquining | parent | next
Claude is perfectly capable of fixing most vulnerabilities; if that's what maintainers want, they can just ask for it. If the concern is cost, Anthropic offers a Claude for OSS program.

The reason Anthropic isn't proactively sending patches is, I suspect, that those patches would not be welcomed, because trust in the models isn't currently at the point where maintainers are willing to merge them without human review, which takes approximately as much effort as writing the patches themselves. You can imagine the wailing and gnashing of teeth about "slop PRs" and "extractive contributions" that'd be all over HN.

eqvinox | root | parent | next
Anthropic's oss-scanner includes patches. We've received a batch and I'm working on some of it.

A good amount of the patches is straight garbage. It's not helped by the fact that a good amount (20~30%, maybe more) of the issues is also junk themselves.

I'll hopefully be able to share numbers (or data) after we're done with the batch.

loading story #50449452
loading story #50449095
matkoniecz | root | parent
> If the concern is cost, Anthropic offers a Claude for OSS program.

thanks! It turns out that I am apparently qualifying for it and I was unaware

lets see whether Anthropic agrees

wonnage | parent
Could also be that most of the findings are low signal noise. Even the batch that was sent to curl was one medium and 39 low severity. One can only assume that the vast numbers of findings that haven’t been sent are even lower severity.
grim_io | root | parent
I'd be careful to take an examplary projects like curl and assume that level of quality is the average or even close to it.

Compared to curl, even the Linux kernel is garbage, and the Linux kernel is probably higher quality than all of the enterprise software that runs the world.