I wish each quant was benchmarked on the same tests as the original network so we could compare their performance
Unsloth publishes KL divergence numbers which measures how much the quantised probability distribution changes vs unquantised: https://unsloth.ai/docs/models/qwen3.8#quantization-analysis
It's a bit bare at the moment, I assume they are going to add further detail later (eg comparison to other quants), similar to their other releases.
loading story #49302519
KL divergence is nothing close to a replacement for benchmarks. As flawed as benchmarks are, KL divergence is a barely useful signal. The fact that Unsloth only just started publishing KL divergences shows how unserious the quantization space is.
loading story #49301832
loading story #49302433
loading story #49301509
That's not a replacement for benchmarks
loading story #49301938