Why do you say API based llms looking iffy at best? Do you just mean current profitability due to market pressures from some companies’ subsidized investor money?
Surely, even if you’re just using open weights models, it should theoretically be cheaper to use them in a highly optimized cloud architecture(even with vendor markups) rather than each person serving their own models from much less efficient (and more importantly, much less consistent volume) self-owned “server under your desk”?
LLMs are becoming commoditized, which means the margins are trending to zero. It's a lot less exciting to spend another trillion on a new model if you can barely make any profit. Meta getting out of the game might be the smarter move.
Companies are going pretty quiet about costs, but I see no reason to believe the cost to train a model exceeds $5B. Moonshot AI's entire funding is like $5B, their $300M revenue is negligible but not nothing. Renting the compute to train a model like Kimi K3 cannot possibly exceed $5B and is probably under $1B. It's probably at least $100M, but also plausibly not. I don't think it's likely Meta would be giving away models for free if the compute cost to train them was in the billions. This 30B parameter is tiny, that's not billions of dollars, that's likely millions, maybe even less.