Hacker News new | past | comments | ask | show | jobs | submit
throwawayffffas | on: AirLLM 70B inference with single 4GB GPU
Assuming 30% gpu power utilization because of all the loading and unloading 29.2 kJ per token
donquichotte | parent
Damn 15 AK47 bullets per token