Hacker News new | past | comments | ask | show | jobs | submit
semessier | on: Do transformers need three projections? Systematic study of QKV variants
V being collinear is obvious, the question is/was also which additional orthogonal projections such as camera position for vision would improve the transformer.