Hacker News new | past | comments | ask | show | jobs | submit
aliljet | on: Qwen3.8-Max: A New Bar for Coding and Cowork
I'm trying and failing to find value running a potential Qwen 3.8 27b dense model on a 16 core, 128 GB of ram, 2080ti box. Yes, the GPU yells for help, but the problem is that no math works to upgrade this machine even when pouring $200 in rent every month into the large model providers...

How are you all justifying economical use of these local models right now? What's the cost efficient way to do this and do better (even with models evolving over time and losing now vs later) than the big labs?

aurareturn | parent | next
You will simply not get more value out of running a local model vs paying for a subscription/API from the cloud in 2026. There is no math that will make local models come out ahead in $/intelligence/token.*

The point of local models is privacy, offline use, and maybe no guard rails.

* Not talking about enterprises that buy DGX racks and host Chinese models for internal use.

loading story #49151425
loading story #49152806
loading story #49151990
dpc_01234 | parent | next
There's no point, except if you want privacy and independence. I am playing with a personal assistant that checks my emails, calendars, sends me an agenda and maintains my TODO list. I am not sending such data outside.
loading story #49156258
loading story #49156057
jbellis | parent | next
The only scenario is if you have enough work to do batch inference. Using a tiny fraction of GPU capacity to decode a single request at a time just doesn't make sense, as you say.
AureliusMA | parent | next
If you can do inference on the CPU, drop the GPU : it should be faster.
bsder | parent
> $200 in rent every month into the large model providers

We all know that is hugely subsidized, and I guarantee that OpenAI and Anthropic are looking to enshittify that ASAP.

The enterprise users, however, are not subsidized like that. They pay per token. And some developers in those companies are chewing down a lot of tokens. Self-hosting an open weight model could be a massive savings very quickly. It also gives them negotiation leverage when talking to OpenAI and Anthropic.

loading story #49152788