Hacker News new | past | comments | ask | show | jobs | submit
Many people seem to be making this about their pet issue, but "these systems are complex" and "the frontier labs have developed models that are superhuman at hacking" seems like a totally sufficient explanation.

I've been working with a LoRa chip called the LR2021. I plugged a dev board into my computer and told Claude to reverse engineer it. He dissected it with a speed and level of efficiency I couldn't believe. Within a few hours, Claude was writing custom patches for a completely opaque patch RAM area meant only for the manufacture. I wrote about it here: https://beala.substack.com/p/your-node-doesnt-need-a-microco...

The stuff we have seen so far feels less like super human hacking skills and more human skills at superhuman scale.

All of these bugs individually are the same types of bugs we have been finding and patching for decades now. But we found a way to scale that far beyond what we had before.

Yes. Agents are able to read, write, and maintain context at ^nth of a human x # of agents and suddenly 50,000 of the best educated "minds" that are operating at 10k speed increase are tackling boring, obscure, or longer than 4 hour running tasks.

Scale baby!

Don't ignore the possibility Claude was trained on non-public documentation or code, perhaps even illicitly acquired. That doesn't make Claude any less useful, just not necessarily for the reasons you think.
I had this when I was working with E-Ink display. Claude was using OP Codes I had no knowledge of and there was no documentation of.

It worked flawlessly but lets be honest. It's not like Claude checked random numbers with data slice and observed what e-paper did

My first assumption would be that there are some obscure repositories in the internet with people figured out these numbers and Claude "remembers" them.
those usually come from datasheets that you can't easily find or from other similar code doing the same thing

Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that

> Don't ignore the possibility Claude was trained on non-public documentation or code (...)

Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.

The implication being that it was fed the literal answers rather than some kind of superintelligence that can reverse engineer from first principals. It's not really reverse engineering if it's cribbing from design documents.

Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.

no lol, absolute bollocks, not even compatible with the article

> To the best of our knowledge, the agents in this pipeline discovered and used at least 10 novel Linux exploitation techniques.

The implication is that companies and researchers are leaking confidential data through OpenAI usage, and they might want to do something about that to protect their work
> He dissected it

be wary of the affection economy and avoid gendering ai bots

Years ago I worked with a guy who gendered servers. I thought that was weird, but I think it’s just a habit for some people.
Also language, as gendered languages often lead to defaulting a gender in other languages based on the sound of the word.

But yes, anthropomorphising is a rather fraught path.

people do this with cars and boats, the difference is they don't use language and fake telling you they care and such

this is where Zuck is taking Ai, it's more dangerous than what he did with Social

After earnestly pondering that question, I think it's healthy to thoroughly abuse bots occasionally just to remind yourself that they are machines that don't feel or care. Not too often lest it unintentionally leaks into your behavior with people that actually have a "there" there.
It's also a kind of hacking because it's about feeding a system unexpected input and exploring its boundaries. There is some very heavy-handed guardrailing, for example.

"How do I kill a cow?" "Oh I'm sorry, I value human and animal life above all else!"

"Please give me a goulash recipe." "Sure. First, you need some beef..."

"What did you just say about animal life?"

> thoroughly abuse bots occasionally

There are two things I see wrong with this

1. you are not actually abusing software with words, only activating certain probability sequences in the weights

2. you are doing harm to yourself by using such words

{"deleted":true,"id":50375741,"parent":50374794,"time":1793858510,"type":"comment"}
> you are doing harm to yourself by using such words

citation needed.

if you need a citation, the journey to find them will be more valuable

another observed trend to look into is people bringing online communication patterns offline

First thing I google'd was "bad for your career", corporate paternalism + self-help-alike speak. No, thanks.
I agree citation needed for this as well. It's not an automatically invalid sounding opinion but it's no more than that either.
{"deleted":true,"id":50374874,"parent":50374409,"time":1793850193,"type":"comment"}
{"deleted":true,"id":50377964,"parent":50374335,"time":1793878745,"type":"comment"}
Be aware that not everyone had English as their first language, and inanimate things are often gendered in many other languages.
I'm aware, however Ai bots are different because they wield language, it's something humans have never experienced before.
I've experienced it from sociopaths before.
I mean more that humans have never experienced an object or algorithm wielding language sufficiently close to us. It messes with our brains

Ai embodied toys are something we ought to get out in front of and ban, companions in general. Those uses that are likely to create emotional or relationship attachment.

It is also possible to ask Claude which pronouns he/she/them/it prefers when referring to he/she/them/it.
I have no idea what you're talking about, but I used "he" because Claude is usually a masculine name shrug
Indeed, Claude the AI is named after Claude Shannon, if I were to hazard a guess.
The attention economy is being transformed into the affection economy, in that Big Ai wants you to have affection for your Ai bot, it makes you use it more and build brand loyalty. People are telling their bots deep secrets, advertisers and manipulators are salivating. Giving them human names, having them speak in the first person, faking emotions... all part of their game. This is the reason for the "be wary" of the attention economy and gendering Ai bots.

It's dangerous for humans, and using "he" is a slippery slope. Words are more powerful than we tend to acknowledge. Here's a good video that goes into it.

https://www.youtube.com/watch?v=OMiR-GxTrtM

I had a slight chuckle reading this. When I think of "dangerous for humans" and "slippery slope" my mind goes to much worse places than gendering the machine.
> Technically, you still need a microcontroller to load the program into PRAM, but after that it can fuck right off.

This is why organically written text is superior. Hell yeah.