Hacker News new | past | comments | ask | show | jobs | submit
I’m also not an expert but that sounds a lot like the toilet seat problem. Mathematically it’s preferable to just always leave the seat as you used it, instead of using effort to return it to some desired state. This minimises effort in aggregate for all parties. I think it’s possible that routing every request through a simpler model first ends up more expensive in the long run, depending on the cost of that simple model. And if it’s too simple, the routing inefficiencies and mistakes will produce worse results.

I think one of the more difficult problems with this is that the complexity is often unclear at first. It’s not until some thinking and analysis that the complexity emerges. It’s just not possible for a simple model to deduce this, and we don’t want a dumb model investigating a complex issue.

I think in the medium term we end up with large models which are efficient with simple queries. Meaning very few tokens used to answer simple queries, keeping costs low. So one just uses this large model for everything.

Longer term I think LLMs move closer to the structure of the brain, whereby different regions/models have specialised functionality, and they communicate and coordinate depending on the task. For example, telemetry doesn’t need to pass through the prefrontal cortex. It just goes straight to the telemetry model which is super simple and cheap.

> Mathematically it’s preferable to just always leave the seat as you used it

Ah, but the consequences of a female type partner encountering an up seat in the middle of the night far outweigh my consequences of having to raise the seat during the day.

This is a good point. If your partner is blind and disabled the extra effort may be warranted.
She is not. But I do afford that 2am bathroom calls (normally not ones I myself answer) approximate the state.