> Switching LLM providers as a company is not, once all your employees have it integrated with GitHub, their IDE, etc
I don't think this is entirely true. Companies are deploying open-source LLM gateways to be able to do that very easily now. Today, you can switch Claude CLI to any model besides Anthropic via https://github.com/BerriAI/litellm and there are major companies doing that.
This works with your favourite ADEs such as Orca as well so I disagree with this premise. The whole experience of switching is pretty seemless based on personal experience.
We saw this play out in the cloud era, notably with OpenStack. Even through the core abstractions (chat for LLMs, compute and storage for clouds) can be almost perfectly abstracted over, providers will race to add niche, enterprise-y features that are harder and harder to put behind a gateway or proxy. Those features aren't the core value of the platform but they are important enough to create lock in.
And if the LLMs are as powerful as the very same providers claim it will be trivial to write code/configurations that bypass lock in.
Agreed, deploying AI gateways has become extremely common in enterprise settings. A lot of admins think they are going to use it for guardrails, but I think the real value is in centralized cost tracking and ability to (relatively) seamlessly aggregate providers.
AI can program a cost tracker tool pretty easily.