The value of money, and the ability for average people to have influence on future events, ultimately is tied to the fact that the labor of an average person has inherent value.

Once that is no longer true, all bets are off: All promises of compensation become meaningless, and it's easy for those in power to rationalize to themselves why they no longer are beholden to their earlier promises. They will say to themselves "The situation ended up more nuanced than we expected"

(Which is exactly what Google said in February 2025 when they reneged on their pledge not to use their AI for military applications)

So perhaps they would only respect money and property up to the point that it's necessary.

The more realist answer is that they will keep respecting money and property as capital works to reinforce the status quo and enrich shareholders. You think just because you can do your job faster--or completely--with a machine that you have to work less? Hardly! Now this is where interesting policy questions spring up but I digress.

https://archive.is/20251015175242/https://www.bloomberg.com/...

As long as they're still looking for prompting customers, they're clearly not there yet. There is some solace in that.

I think this is the major hole in the argument, and in the "AI maximalism" position you present. There is no such thing as an omnipotent genie, or a magic eight ball that can answer any question. Any technology has limitations, and if you assume as a premise that the technology does not have meaningful limitations (or that the limitations are irrelevant, which is really what you said), the conclusions you draw will also make no sense. This is how you get nonsensical ideas like the singularity.

because they don't control the physical paper that is money.

I dont understand what you mean by "fiat tokens". I don't imagine most people will accept tokens as a currency because they are as good hold of value and predictable as bitcoin.

In a token economy, certainly what you say is a possibility, but as you can imagine, that economy will be unsustainable so back to "regular" money.

So why would you allow that kind of power to be concentrated in the hands of a few companies? If the state can use this technology, which keeps improving on its own, to manage an entire nation's resources better than any human or group of humans, why would we still need private companies to do it? Especially when the ones with the most resources/capital would be able to just win with everyone else and accumulate all the existing resources.

But the bigger issue lies elsewhere. When you replace human labor, you also replace soldiers with robots. Until now, a leader with autocratic tendencies couldn't easily order the military to turn against domestic "terrorists" who are actually their political opponents. But once armies are replaced by robots, that could all change, and we could see democracy go into decline.

Doesn't that factor into the future revenue? Feels like people imagine suddenly fully owning the means of production in a vibrant economy but how does that really make sense?

If Microsoft Windows and Office can have a valuation in the trillions then Claude eventually working its way to the same ballpark isn’t a crazy assumption.

You have to wonder how many people in AI leadership positions are already making decisions based on this expected outcome.

"AI" doesn't pump/treat sewage or drinking water, grow crops, mine metals, change wound dressings, deliver babies, stock grocery stores, drill teeth, walk dogs...

It's not the replicator from Star Trek!

AI maximalism is a Silicon Valley religion. You don't need anything close to it for some of these valuations to work. (Anthropic would be worth ~$500bn at Google's IPO multiple on revenue [1]. Anthropic is currently growing 6x faster.)

To be clear, I think these valuations are rich. But the notion that AI maximalism is required to sustain them is just false. $1.5tn sits comfortably on about $60bn of earnings, which with Anthropic's gross margins requires on the order of hundred of billions of dollars of revenue. That's super ambitious. But not Anthropic is the economy ambitious.

[1] https://news.ycombinator.com/item?id=50247011

[2] https://www.multpl.com/s-p-500-pe-ratio

Really the only scenario where Anthropic could screw over its investors in the way you're describing is if it could also take over the legal system and modify its own contractual obligations. Which, I mean, I guess anything is possible if we are in AI maximalist-land. But "AI is the law now" would be a much bigger story than "investors frustrated with low dividends."

If the economy is planned based on datasets, which are inherently backward looking then our best option is our ability to surprise. But that’s for the artists and philosophers to figure out.

1. the goal of ai is to end capitalism by disrupting scarcity in economics

2. if ai can build a closed system that prints ai factories, the 99% of humanity essentially are just genetic diversity live stock.

not an original thought, ive seen Her

suppose this is just another way to say, "wtf are we doing?"

The closer you get to the real or plausible "omnipotent genie", why would the person in possession of such a thing respect the rights of the formal owner - and why the engineers and operations managers respect the right of the management - this along with the question why the maybe-conscious super-intelligent machine itself would respect the rest of the hierarchy?

For now, these structures actually need massive resource inputs and that doesn't look to change soon but naturally one should be on look-out for the possibility of the tide shifting.

Lots of other fun risks (highly recommend checking that section) but not that one :(

We don't have democracy because somebody thought it was fair or just for people to rule themselves. We have democracy because in the past, people got angry, picked up rifles and shot enough authoritarians (and their submissives!) for the rest of them to understand the group with more people has more firepower and tends to win.

So now we periodically check which group has more people with ballots instead of bullets.*

With autonomous drones and robots, democracy stops being the local optimum.

And once we slide (are pushed off) into a new more stable state (pun not intended), we won't have the (fire)power to get out of it.

* actual AI, which does exactly what it should correctly, not a statistical model of behavioral patterns

* but indirect democracy is expressing a point in n-dimensional space (your opinion) by selecting one of a predefined set of points (parties/politicians available) - which is stupid - and the cracks are already starting to show as e.g. nobody wants more surveillance, yet we keep getting more of it in both EU and US

---

The "permanent disempowerment" people keep talking about won't be a "rogue" model, it'll be doing exactly what told by the rich and connected.

EA-aligned AI labs share this exact ethos. In their view, fiat money and equity markets are merely temporary scaffolding used to raise capital until AGI is achieved. Once a central superintelligence controls manufacturing, software, and logistics, resource distribution shifts from market supply-and-demand to algorithmic optimization.

[0] https://digital.library.cornell.edu/catalog/ss:34227574

[1] https://en.wikipedia.org/wiki/Joshua_N._Haldeman?hl=en-GB