Hacker News new | past | comments | ask | show | jobs | submit
nico | on: Edge0-145B: Frontier AI Capabilities on Your Own Computer
It's amazing the level of optimizations we are discovering. At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago

Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well

utopiah | parent
> At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.

CSSer | root | parent
I guess the question is less about what kind of model is viable and more about what kind of application fits any given model. I'm highly skeptical that my employer is willing to pay long-term costs of close to double what I make an hour for token usage, high precision or otherwise. I'm also highly skeptical of the return in some cases. I have to put my codebases in straitjackets tighter than ever to keep them on the rails, and this requires yet more tokens to do so. For premium work, okay, maybe? But it doesn't always excel at that work. For typical work, I don't see it being worth it for the big models.