Hacker News new | past | comments | ask | show | jobs | submit
Given how "smart" some of the 26b dense models are now, I would not be surprised to see a strong 40b MoE.