Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 1 - Transformers
- Source
- youtube.com
- Author
- Stanford Online
- Date
Context
For a learning to predict the next , a unit of text such as a word or word fragment, training must hide the continuation it is supposed to predict. The CME295 lecture deck explains causal masking: future tokens to be decoded must be hidden at training time, so the decoder's cannot read later positions.
The deck says the benefit is that all operations are vectorized, meaning the calculations for many positions can be done together. Read that way, the mask is what lets positions be processed together without exposing later answers to earlier ones. The deck's decoder slide shows the masked attention step inside the decoder, which an earlier slide labels as the part that generates the next token. This applies to next-token prediction, not to every text-generating transformer design.
- transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
videoStanford Webinar - Building Trustworthy AI: Navigating Security in the Agentic Era
videoStanford CS547 HCI Seminar | Autumn 2025 | HCI Systems in the Age of AI Code Generation
videoStanford CS25: Transformers United V6 I Advancing Science and Medicine with Collaborative AI Agents
videoStanford CS153 Frontier Systems | Andreas Blattmann from Black Forest Labs on Visual Intelligence
articleA Component-by-Component Walkthrough of How Transformer LLMs WorkAndrew Ng
videoStanford CS25: Transformers United V6 I On the Tradeoffs of State Space Models and TransformersStanford Online
videoStanford CS25: Transformers United V6 I From Language Models to Native Multimodal IntelligenceStanford Online
Checking sign-in…
Loading comments…