Hacker News new | past | comments | ask | show | jobs | submit
These nerf trackers do not work. I wish people would stop posting them.

They don't detect a quantized model until the quantization gets down to 4 bits or less.

Humans absolutely do detect quantization well before 4 bits...

Try it yourself if you want. Try a small local model, both quantized and not, and I would be very surprised if it isn't immediately obvious which is which.

But quantized local models also dip in evals.
loading story #50431824