• attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.

The compromise nearly every current model ships: grouped key-value heads give you multi-query's small cache without the quality drop, and the paper shows you can convert an existing multi-head checkpoint rather than retrain from scratch.

• articleFast Transformer Decoding: One Write-Head is All You NeedNoam Shazeer
• articleSome Intuition on Attention and the TransformerEugene Yan
• articleA Visual Guide to Attention Variants in Modern LLMsSebastian Raschka, PhD

Sign in to comment.

Loading comments…