A Visual Guide to Attention Variants in Modern LLMs
Source
Sebastian Raschka, PhD
Author
Sebastian Raschka, PhD
Date
Key takeaways · AI-distilled
attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → was invented to fix an RNN bottleneck. An encoder squeezed a whole sentence into one hidden state, so the decoder could not reach back to the specific source words it needed; attention lets each output position read the full input directly.
Q, K, and V have distinct jobs: the query is what a tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → is looking for, the key is what each token advertises for matching, and the value is the information mixed in once the weights are set.
The attention matrix is not a stored object. It falls out of QK^T passed through softmax, one row per token, with future positions masked off in a decoder-only model.
GQA, or grouped-query attention, keeps every query head but has several of them share one set of key/value projections. Fewer KV heads means a smaller KV cacheThe memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.Full definition →, so it is a memory win that leaves the rest of the decoder recipe alone.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
Why it matters
If you want to understand how modern LLMs actually manage attention — and why models are shifting to GQA, MLA, and sparse/hybrid schemes — this visual breakdown maps each variant to real 2025/2026 architectures so you can reason about model efficiency and design tradeoffs concretely.
Key quotes
“In my opinion, hybrid architectures are still a novelty, and the main selling point is mainly (long-context) efficiency versus just modeling performance.”
Sebastian Raschka
“I find that I get better tok/sec throughput when running LLMs locally using more classic setups like GPT-OSS with grouped-query attention.”
