Hacker News new | past | comments | ask | show | jobs | submit
zozbot234 | on: Memory has grown to nearly two-thirds of AI chip component costs
MoE is fine. You can put the shared weights on the 5090 (will fit handily even for the largest models) and expert weights on CPU, possibly with weights offload from storage.