I think a new default free model for AI Mode being so close to the frontier will be a massive jump in perceived AI intelligence by the general public.
It will be the same as these smaller models from OAI and Ant. Strong in benchmarks but pretty bad in real world use.
I've read this same comment every time a new Luna came out, and the difference between free GPT (Luna) and paid (Sol and Astra) is still outrageous for general use.
loading story #50471937
It's definitely not the case that small models only do well in benchmarks. We're working on a battle system on top of OpenCode, so have data for real people doign real work.
Right now DeepSeek Flash 4.1 Flash has a 64% win rate against GPT 6.1 Sol. Small data still, but it has 33 wins vs. 18 losses, which is confidence interval of 52% - 78% win rate. https://agentduel.com/leaderboard/models/deepseek/deepseek-v...
loading story #50473944
loading story #50471924