Hacker News new | past | comments | ask | show | jobs | submit
A hosted instance serves multiple customers at a time. A local model only one.
How many though? At 1m context you quickly fill a full gb300's 280gb of memory