Do transformers need three projections? Systematic study of QKV variants

arxiv-logo-fb

Computer Science > Machine Learning arXiv:2606.04032 (cs) [Submitted on 1 Jun 2026] Title:Do Transformers Need Three Projections? Systematic Study of QKV Variants Authors:Ali Kayyam, Anusha Madan Gopal, M Anthony Lewis View a PDF of the paper titled Do Transformers Need Three Projections? Systematic Study of QKV Variants, by Ali Kayyam and 2 other authors View PDF HTML (experimental) Abstract:Transformers have become the standard solution for various AI tasks, with the query, key, and value (QKV) attention formulation playing a central role. However, the individual contribution of these three projections and the impact of omitting some remain poorly understood. We systematically evaluate three projection sharing constraints: a) Q-K=V (shared key-value), b) Q=K-V (shared…

Continue reading at Hacker News arxiv.org
Advertisement