> What is the hardware cost to run a single instance of a frontier model?
The latest Kimi model, which scores higher than 5.1 Fable on Artificial Analysis, is estimated to require around 6-7 TB of VRAM to run. If we are going with many A100 GPUs for this, then we are looking around a million dollars in GPUs alone
What? We know exactly what it takes to run: 2.8 TB VRAM for FP8...1.4 TB VRAM for FP4.