> In software engineering, we learned a long time ago that giant monoliths are not the answer.

That depends on the question. If the question is "How do I utilize 10,000 software engineers?" then yes, a monolith is not the answer. However, this isn't a problem most companies face. Starting out with microservices is rarely a good idea either as you'll have a hard time getting the boundaries right.

The ability to do "everything" is an emergent property of how they are built and trained.

If you want a model that can do X but don't care about Y And Z, intuition says you only have to train the model on data relating to X. However, early LLM research showed that doing this results in a far less X-capable model than one trained on data for Y and Z as well.

I was thinking about this recently as well. Smaller, specialized models that can run on device just make so much more sense to me than these huge generalized models that need huge infrastructure run.

I see it as analogous to CPUs and specialized coprocessors. You don't want your general-purpose CPU bit-twiddling your USB data lines or decoding ethernet frames or blitting pixels. Specialized hardware does it so much more efficiently. And yet, powerful general-purpose CPUs remain common. Partly that's because there's a lot of complicated stuff being run that just isn't suited for specialized hardware; try making a C++ compiler coprocessor and see how far you get. And partly it's because that powerful general-purpose CPU is always present and can be instantly reconfigured to do anything. You'd rather use a dedicated cryptography coprocessor than implement your crypto on the CPU, but doing it on the CPU allows you to implement new crypto algorithms quickly. What's better: doing AES on a coprocessor on the CPU? Well, that depends on whether you have an AES coprocessor.

LLMs are the powerful general-purpose CPU of the ML world. Using an LLM as a classifier is woefully inefficient compared to using a dedicated classifier. On the other hand, that LLM exists right now and you can hook it up in ten seconds. You could ask: why use an LLM when a dedicated classifier would be so much more efficient? You could also ask: why build a dedicated classifier when a sufficiently capable LLM is available off the shelf? Neither one is the right answer every time.

I am not sure a best practice in one area translates nicely to another. LLM's power comes with it being a monolith.

> single model that can do everything

"AGI"