We saw this play out in the cloud era, notably with OpenStack. Even through the core abstractions (chat for LLMs, compute and storage for clouds) can be almost perfectly abstracted over, providers will race to add niche, enterprise-y features that are harder and harder to put behind a gateway or proxy. Those features aren't the core value of the platform but they are important enough to create lock in.
And if the LLMs are as powerful as the very same providers claim it will be trivial to write code/configurations that bypass lock in.