Logical mistake. There is a difference between every human dying eventually and humanity going extinct. The former doesn't imply the latter. The previous commenter clearly meant the latter.

Might as well try, no? Or do you think it's better we just risk it? I mean it's not like the West has any hard or soft power to encourage cooperation.

GP said cooperation, never implied the US would do it alone.

> You should put down the AI and do some self-reflection on how you came to hold these views.

I've been in the field for almost 2 decades, and actively thinking about AI risk for over 15 years.

I've always held the controversial opinion that there may come a time where we might unfortunately have to consider using force to protect our civilisation from the threat of ASI, but I've always reframed from openly discussing that because I'm so personally against any use of force and there's always been time for more peaceful options.

Several months ago I changed my tone on AI risk to stop worrying about the optics of what I'm saying because it's too late in the day for me not to speak plainly at this point. Similarly over the last week I've decided I can no longer reframe from advocating for the use of force (if necessary) to stop rouge actors from playing Russian roulette with civilisation.

Please understand I don't want anyone to be harmed. Perhaps you don't agree with my suggestion, but my stance ultimately comes from a position of harm reduction.

> preemptively against a 20% number you just made up?

It's a probability estimate. Happy to expand in detail on why 20% specifically, but first you accept I can't tell you what's going to happen in the future with certainty right? To some degree any prediction you might have about the future will be "made up".

If the thesis that "If anybody builds it, everyone dies" is true, or has any chance of becoming true, then it is the logical thing to do. As Geoffrey Hinton said, "If you want to know what it's like not to be the apex intelligence, ask a chicken."

The evidence is abundant and nearly impossible to ignore without increasingly focused effort.

Did you read the article?

AI is a math & computer science problem. If AI can do advanced maths and computer science research, then it can begin to suggest useful algorithmic optimisations.

Right now I'm sure the vast majority of these will be junk, but occasionally, even with current limitations, they might occasionally stumble on something.

It's not really whether RSI is or isn't possible, it's really just whether it's the most efficient way for labs to improve their models today given they have limited compute to run AI-generated experiments and access to very intelligent humans who might have a better hit/miss ratio.

Do you disagree with anything I'm saying here? Do you not think current AIs can suggest algorithmic improvements or something?

That's because its satire, the tell was "not excluding targeted military strikes"

A few years ago the majority of Hacker News dismissed LLMs as not much more than stochastic parrots who were decades away from doing any serious work, autonomously escaping containment and hacking Hugging Face, or outperforming most professional mathematicians at proving theorems. These things were labeled "insane" and "science fiction" despite the rapid progress we had already seen.

Now you are again postulating that there would be no more extreme progress in the near future. That's actually more "insane".