Hacker News new | past | comments | ask | show | jobs | submit
xnorswap | on: AirLLM 70B inference with single 4GB GPU
I wonder what this measures in J/token.
throwawayffffas | parent
Assuming 30% gpu power utilization because of all the loading and unloading 29.2 kJ per token
loading story #49156917