GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Source
arxiv.org
Author
Joshua Ainslie et al.
Date
Why it matters
The compromise nearly every current model ships: grouped key-value heads give you multi-query's small cache without the quality drop, and the paper shows you can convert an existing multi-head checkpoint rather than retrain from scratch.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
