Hacker News new | past | comments | ask | show | jobs | submit
foldl2022 | on: Do transformers need three projections? Systematic study of QKV variants
Gemma-4 E2B/E4B models reuses K-V cache from other layers, which do things in a "transposed" way: not reuse Q/K/V matrices within a single layer, but reuse across different layers.