Hacker News new | past | comments | ask | show | jobs | submit
K0balt | on: Flash-MoE: Running a 397B Parameter Model on a Laptop
My thoughts exactly. Something like this could make it so that modest GPU capacity, like a pair of 3090s , and lots of RAM could make big inference more practical for personal labs