Hacker News new | past | comments | ask | show | jobs | submit
Microsoft (MSAI) showed that quality >> quantity with their Phi model a couple of years ago. It was a small model, but trained on very high quality data, and did better than much larger models.
Phi was massively overfitted on the data and didn't generalize well in practical use though.