Stanford CS25: Transformers United V6 I On the Tradeoffs of State Space Models and Transformers
Source
youtube.com
Author
Stanford Online
Date
Why it matters
Gu lays out where state space models and transformers each win, which informs choices about long-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → efficiency, hybrid architectures and inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → cost.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
videoStanford CS153 Frontier Systems | Andreas Blattmann from Black Forest Labs on Visual Intelligence
videoStanford Webinar - Building Human-Centered AI: From Reward Functions to Real Products
videoStanford CS547 HCI Seminar | Autumn 2025 | Building Control and Trust in Human-AI Systems
videoStanford CS329A Self-Improving AI Agents | Part 8 | Agentic Evaluations and Long Horizon Tasks
videoStanford CS25: Transformers United V6 I From Language Models to Native Multimodal IntelligenceStanford Online
videoStanford CS25: Transformers United V6 I From Next-Token Prediction to Next-Generation IntelligenceStanford Online
video2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]Latent Space