> But I do not see a way to rule it out as a possibility.
Your argument is unfalsifiable. Anything can happen by this logic and they could be valued whatever trillion $, like, maybe they’ll invent a way to travel through the multiverse. Maybe they’ll create the star trek replicator thing.
It is certainly not unfalsifiable. There are other scenarios in which this does not happen. For instance, the most obvious is that other organizations develop a similar capability at roughly the same point in time. This results in a "multipolar" economy that is mostly controlled by a small set of distinct owners of this kind of capability. The multipolarity prevents total capture and thereby prevents the rest of the scenario. Another obvious scenario is one in which the government acts soon enough to short-circuit this process before one organization gains a sufficiently decisive advantage that government intervention is no longer efficient enough to be effective. Yet another obvious scenario is one where a short-term financial crash deprives Anthropic of enough capital that this strategy is paused or slowed down enough to allow it to be countered. And yet another obvoius scenario is one where the scaling laws simply stop scaling or the amount of compute needed to scale them to this point cannot be physically obtained, thus allowing time for the strategy to be countered.
All of these alternatives would come with clear data points early on which would allow one to distinguish them from the original scenario I described. Right now, though, the data is ambiguous, and some data points lean slightly in favor of what I described, but not nearly enough to make any guarantees whatsoever.