at their scale they could also just run a large on-premise or rented (basically still cloud, but cheaper) GPU cluster and run through that. fixed costs, even license a SOTA model’s weights if you’d like
The problem isn't really Uber, Microsoft or Nvidia, it's all the smaller none IT companies that also have developers on staff. They are screwed. $1500 per seat per month is just way to expensive, but they also can't afford to build and maintain their own on-premise solution. If Microsoft can't afford to run CoPilot for their own developer, what chance does any of their customers stand?
If the large, well founded IT companies in the world believes the current AI cost is to high, then Anthropic, OpenAI and CoPilot have no actual customer base. AI is then relegated to very profitable niche business, but that can't fund the R&D for the models.
It's an extra 18k a year for developer tools when they're paying how much a year per developer? Having software developers at all isn't cheap.
Also, I don't believe you need to spend $1500 a month on a coding agent if you optimize usage at all.
loading story #48391259
loading story #48387967
loading story #48388373
There's models for every price point. What was SOTA and stupid expensive to run a year ago is a cheap flash model today.
Why are smaller non-IT companies "screwed" because they can't pay out the nose for their developers' AI usage? They're non-IT companies, developers are presumably not on their critical path, or not their bottleneck. Developers can keep on writing code the old way, or doing it with a more reasonable AI spend. I don't see how this "screws" any company.
loading story #48388214
> even license a SOTA model’s weights if you’d like
Yeah, I bet all labs releasing SOTA models are more than happy to remove the main way they make money and let you run it locally, especially if you're a big spender like Uber who seems very willing to throw money into the sea as an experiment.
That's going to stop eventually, and I think at that point we're going to see business models more like the major CAD providers.
{"deleted":true,"id":48383807,"parent":48383723,"time":1780493513,"type":"comment"}
I don't think they'll have a choice, open weights models are not far behind. At some point it's essentially a commodity game
loading story #48383920