Vibeleaderboard
← All Intel
Intel / video

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 1 - Transformers

Source
youtube.com
Author
Stanford Online
Date

Context

For a learning to predict the next , a unit of text such as a word or word fragment, training must hide the continuation it is supposed to predict. The CME295 lecture deck explains causal masking: future tokens to be decoded must be hidden at training time, so the decoder's cannot read later positions.

The deck says the benefit is that all operations are vectorized, meaning the calculations for many positions can be done together. Read that way, the mask is what lets positions be processed together without exposing later answers to earlier ones. The deck's decoder slide shows the masked attention step inside the decoder, which an earlier slide labels as the part that generates the next token. This applies to next-token prediction, not to every text-generating transformer design.

Terms in this piece · Glossary
  • transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
  • attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…