Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU
https://www.neomindlabs.com/2026/06/08/running-gemma-4-26b-at-5-tokens-sec-on-a-13-year-old-xeon-with-no-gpu/I am running Qwen3.6-35B-A3B locally on my 16GB mac with 7-9 tokens/second. Link - https://github.com/deepanwadhwa/samosa-chat
This is a GPT4 level model running locally with a decent speed on a 16gb ram macbook air.
But you said 7-9 tokens/second, that's not a decent speed. I'm not an expert by all means but in my local experiments, less than 12 to 16 tps is too slow.
https://mikeveerman.github.io/tokenspeed/?rate=10&mode=text
You think of an idea that you want to have the LLM process, queue it up, and go back to what you were doing. Once you've finished reading the next article on HN about a 5 tps Xeon, your task will be complete. It's kind of like using a 3D printer: It doesn't matter if a print takes 10 hours, because when you come back in the morning it will be done.
Yes, with top-tier GPU farms you can hit hundreds of tokens per second. But if the old Xeon in the closet can get useful work done at 5 tokens per second, there are lots of people and lots of use cases where a free, unlimited 5 TPS stream is worth more than paying a dollars per day to get access to a 500 TPS source.