Hacker News new | past | comments | ask | show | jobs | submit
myshapeprotocol | on: AirLLM 70B inference with single 4GB GPU
Running 70B on a 4GB GPU is wild. Really impressive engineering feat for resource-constrained environments.