Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.
But why would you use a model to store factual knowledge, that is stupid. We want intelligence, not a database.
Models cant make any decisions if they have 0 idea that the feature exist in this language or in some general fact.
For example you would tell a model hey, become an expert in this language for me, search it online, it would still need to learn it and download the data to it's context and then increasing the memory usage, there's no way around it.
that is why we enable web search for the agent. the memory can come from the internet.
deepseek-v4-flash needs web search to return true facts.
Yeah but results are worse, see https://news.ycombinator.com/item?id=49301574