I've been in the field for almost 2 decades, and actively thinking about AI risk for over 15 years.
I've always held the controversial opinion that there may come a time where we might unfortunately have to consider using force to protect our civilisation from the threat of ASI, but I've always reframed from openly discussing that because I'm so personally against any use of force and there's always been time for more peaceful options.
Several months ago I changed my tone on AI risk to stop worrying about the optics of what I'm saying because it's too late in the day for me not to speak plainly at this point. Similarly over the last week I've decided I can no longer reframe from advocating for the use of force (if necessary) to stop rouge actors from playing Russian roulette with civilisation.
Please understand I don't want anyone to be harmed. Perhaps you don't agree with my suggestion, but my stance ultimately comes from a position of harm reduction.
> preemptively against a 20% number you just made up?
It's a probability estimate. Happy to expand in detail on why 20% specifically, but first you accept I can't tell you what's going to happen in the future with certainty right? To some degree any prediction you might have about the future will be "made up".
There are dozens of escalating, yet fundamentally less destructive means to prevent a data centers from continuing to operate (if that's the goal). Not starting at those more modest interventions is political malpractice and in opposition to your harm reduction narrative.
Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
If you're referring to the "nuke the data centers" line in my bio, that's tongue in cheek. It's an expression of my sentiment towards AI at the extreme.
To be clear, I don't think any country should seriously be considering dropping nukes on data centers. Targeted strikes on data centers operated by rogue private companies or nation states is acceptable to me – assuming pre-warning of intent and appropriate regulatory foreshadowing.
I suspect we agree this is reasonable?
> Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
> Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
No, they're not based on unbounded exponential growth curves at all... I'm actually somewhat skeptical of claims that there could exist an intelligence that far exceeds our own. My assumption is really just that AI will be far faster, far more knowledgable, far more persistent and marginally more generally intelligent than the average human. The first three are already true, the forth we're getting to.
We're on an exponential curve right now and I suspect the rate of progress will accelerate some more as a result of RSI. This will likely continue for 2-3 years (if we have that long) before starting to flatten out.
As for my doom prediction, my median concerns are around around the destabilising nature of ASI.
If it can be controlled then it will concentrate power like never before, and I believe this will destabilise civilisation. I can go into more detail here if you wish.
However, I'd still put these destabilisation risks at under 50% in total. My concern more broadly is the majority of ASI outcomes are doom scenarios. It's extremely hard to imagine a world in which you have a magic box that can grant any wish that intelligence can grant, but which we have a functional civilisation. You have to basically assume all potentially dangerous wishes won't be granted and probability the only feasible way to do that is restrict who can make the wishes – but then you have the concentrate of power risk.
I largely agree with the premise "If Anyone Builds It, Everyone Dies", so we must try to avoid building it. I think we can build ASI safely, but we're clearly not on the path right now and I suspect it might take decades (or more) to figure out how we can build ASI with an extremely high probability of a good outcome.
> Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
I do this. I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
I don't think we realistically have a chance of diverting from this path at this point. I have made peace with this but it makes me very sad and haunts my dreams almost every night.