won't help if they also include trash internet text in the training set as well.
it's kinda known that the gemini models write better english which goes against the "trash data" remarks.
Have you ever tried the Gemini models? They are absolutely atrocious. Like, hilariously bad. One time they kept asking me for "Harriet! Where is Harriet?!" and another time it responded in chinese.
They were not trained on good data.
And higher-quality source material results in higher-quality emergent behavior, everything from thinking to tool use.
Seriously people, read their copyright court case. It literally states why Anthropic focused on books and higher quality material.
Google has to learn to not use trash internet content if they want to catch up.
(note that I haven't tried Neon yet so maybe they learned their lesson and fixed it. All my experience is from 3 months on their Ultra plan and previous models from earlier this year)