Hacker News new | past | comments | ask | show | jobs | submit
There are a lot of real world specialized tasks you can't get better at by logical reasoning or role playing as a human. How you conjugate in some small southeast Asian language. What handwriting looked like in 18th century Norway. These are the tasks which first degrade when you shrink a model. Google's models, in my experience, do well on them, other models not so much.

It makes sense too, because of course Google should have a ton of exotic training data. It's probably their comparative advantage, over getting super-clever with reinforcement learning.