Hacker News new | past | comments | ask | show | jobs | submit
jedbrooke | on: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
the “Actually… But wait!” style responses are so annoying, even Claude opus struggles with this so I’d be interested if meta has done something to cut down on that while still giving good responses
dofm | parent
It's not perfect but it is very terse! Better than BottleCap managed to do with post-training Qwen in ThinkingCap.

I suspect it will help a lot with enabling preserve-reasoning, because the biggest apparent limitation of this model is the 128K context window.

Though the practical issue I am seeing on my M1 Max MBP is that performance suddenly drops off a cliff if I have DFlash enabled.

hadlock | root | parent
128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that
0xc133 | root | parent | next
I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to me on a single Radeon R9700.

I added these arguments to my llama-server call, using the official GGUF release from Meta’s account on HF.

`--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 131072 --override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144`

dofm | root | parent
Useful stuff, thank you. And more reading for me to do!

(DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).

hadlock | root | parent | next
In my experience I am getting 23-24 t/s output with dflash off, and it craters to ~9 t/s with it on, miss rate exceeding 50%. And I'm using the same device as stated on their model page/card. We might need to wait for the software to catch up
opensandwich | root | parent
As an extra data point, I have exactly the same issue on a M1 Max as well.
dofm | root | parent
The model card does say 131K+ but I have no idea what scope the + really has in practice. Feels like overclocking; you're-on-your-own territory?

One thing I will say is that its thinking traces are really quite terse. It genuinely seems to spend many fewer tokens on reasoning. So that might help a bit.