We had relative strength, we needed more strength, we built cranes;
there was something like intelligence, we needed more intelligence, we sought to fill the need;
we were rightly fascinated with intelligence, we studied intelligence itself, we wanted to build it...
And then somebody built things that looked like intelligence, and very rightly some said "Oh, now we have to get to the Real Thing with urgency".
It is far more likely that whoever controls a super intelligence uses it to gain more power and inflict far more suffering
I'm extremely bullish on our future ability to make autonomous death an option for anyone.
Unless there are countervailing forces, it will be deployed, capital will hoard the benefits, and everyone else will be told to fuck off.
I don't have faith in any of the AI labs to make hard financial decisions to deploy hypothetical future AGI in a way that's good for humanity as a whole.
There are too many incentives against, including extreme personal financial incentives for key AI lab stakeholders against.
And if we should take anything from tech history, it's that an exceedingly small number of people look fuck-you money in the face and say "Naw, I'd rather do what I believe in."
Post scarcity societies would likely have entirely different dynamics
What was the first thing we did when we digitized media and therefore made it copyable at negligible per unit cost?
Spent a huge amount of time and money to artificially reimplement scarcity via DRM.
Scarcity-based capital interests (read: most) are not going to willingly give up their profit engines.
It's like the old adage: "If you make AGI you just want to watch computers cry"