Hacker News new | past | comments | ask | show | jobs | submit
It feels like Space Race thinking, honestly. If the US delays China and ends up with an AGI god a few years in advance, that can still be a significant advantage even if China can catch up. This especially would hold true if a theoretical AGI could maximally leverage RSI so the improvement curve becomes exponential.
It's much closer to nuclear arms-race thinking.

If the US or China developes an AGI that can leverage RSI, the game is over. It is over in the same way that WW3 is over as soon as the missiles are in the air: it doesn't matter who has the fastest or highest yield missiles, because there is a finite amount of time before there is nobody left to care about the advantages of one side. Except at least WW3 is over in about an hour, whereas an AGI could take years or decades to complete whatever thing it has been misaligned to desire.

AI alignment research is still in its alchemy era, and global cyber security is underfunded and bad. As a species, we simply do not have the tools to deal with AGI that can self-improve.

What stops another power from developing its own AGI a year or two later? Or from catching up once there are two competing RSI AGIs? The Soviets were a few years behind on getting nukes but they still got them.
The first AGI protecting its turf and sabotaging or corrupting the second AI to be its agent.
I think the verdict is still out on whether having an AGI God fly out of your butt would serve as a strategic advantage in the great power struggle.