Attention Is All You Need
- Source
- arxiv.org
- Author
- Ashish Vaswani et al.
- Date

Context
The 2017 paper showed that a translation model could connect information across a sentence through without a recurrent network passing information along one position at a time. Attention compares text representations and combines information according to their relevance. Removing that recurrent dependency allowed more training calculations to run in parallel. Architecture and motivation.
The authors reported better translation results on two tasks alongside reduced training requirements compared with earlier systems. The parallelism concerns training: the paper's decoder still generates a translation one output element at a time, using what it has already produced. That distinction explains how the architecture accelerates learning without making generation a single simultaneous step. Paper, abstract and Section 3.
- transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Checking sign-in…
Loading comments…


