It’s literally a menu in Copilot.
The model is just some REST endpoint you switch out even in a lot of enterprise setups.
It’s weird that people don’t seem to get this. LLMs are the most easily switched-out technology I’ve ever used as an engineer. The input is literally just free-form text.
> It’s weird that people don’t seem to get this. LLMs are the most easily switched-out technology I’ve ever used as an engineer. The input is literally just free-form text.
It depends, if you're automating business processes (honestly probably where the money is), then you'll need a really comprehensive set of evals and moniotoring to ensure stuff doesn't break if/when you change providers.
If you have none of that, then switching will be very expensive (but it was ever thus).
> you'll need a really comprehensive set of evals and moniotoring to ensure stuff doesn't break if/when you change providers
Unless your self-hosting open weight models, you need this even if you don't change providers, since you have no control of what's on the other side of the API.
That just means the business automation is not deterministic, and highlights the issue with LLMs. The solution is not to switch providers but instead make the LLMs generate something that is deterministic given the same input.
loading story #50469648
Yeah, it's the absolutely weirdest thing about all of this. People keep telling me I need to up my AI game yet no matter how advanced the tools get I'm still typing stuff into a text box. My skill is typing English. I was already pretty good at that.