I personally think our academia should be training and curating these kinds of models and the data they are based on, but this is an acceptable second best.
IMO there are much better ways to spend 300 million in research beyond firing up a cluster for 60 days to train on internet content.
loading story #41855080
there's already https://www.goody2.ai/chat
basically no one in the entire world was willing to spend the kind of money on massive compute and data centers that Meta did spend, is spending and will spend. The actual numbers are (I think) rare to find and so large that it is hard to comprehend it.