Every model you use descends from this architecture, and reading it is how the rest of the stack stops being magic: attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → heads, positional encoding, the encoder-decoder split, and why removing recurrence is what made training parallel enough to scale at all.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
