I'm waiting for FP8 quant, preferably from Google.
If you accept the "ggml-org":
https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai...
https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/blob/mai...
Do they run well on vLLM?
Hacker News new | past | comments | ask | show | jobs | submit