transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Why it matters
A thorough, notation-consistent reference on transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition → architecture variants — from attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → mechanisms to positional encodings and efficiency tricks — that helps engineers understand the design choices behind modern LLMs rather than treating them as black boxes.
Key quotes
“Version 2.0 is a superset of the old version, about twice the length.”
“Later simplified Transformer was shown to achieve great performance in language modeling tasks, like in encoder-only BERT or decoder-only GPT .”
