Hacker News new | past | comments | ask | show | jobs | submit
cpfohl | on: AirLLM 70B inference with single 4GB GPU
Obviously needs downloading eventually :).

It seems like this tool saves on both disk space and RAM, then. Classic trade off: speed vs space.

jedbrooke | parent
I mean, if you had fast enough internet connection, you could just stream it over https