Diffusion-based LLMs are a new paradigm for text generation; they progressively refine gibberish into a coherent response. But what's their connection to Transformers? In this video, I unpack how Transformers evolved from a simple machine translation tool into the universal backbone of modern AI — powering everything from auto-regressive models like GPT to diffusion-based models like LLaDA. We’ll go step-by-step through: • How the Transformer architecture actually works (encoder, decoder, attention) • Why attention replaced recurrence in natural language processing • How GPT training differs from diffusion-based text generation • How BERT’s masked language modeling inspired diffusion LLMs • A concrete walkthrough of LLaDA’s masked diffusion process If you’re new here, check out my previous videos for an intuition-driven introduction to diffusion models and how physical diffusion inspired them: https://youtube.com/playlist?list=PL4bm2lr9UVG3SN79Y6WBe4OOlEiO88vie&si=RcTREWUyVSAZRriv 📚 Free slide deck: https://patreon.com/juliaturc 📚 Papers: • Original GPT: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf • BERT: https://arxiv.org/abs/1810.04805 • LLaDA: https://arxiv.org/abs/2502.09992 ▶️ My previous video on Transformers: https://youtu.be/LE3NfEULV6k?si=SAaHbw6jD14nc7IM 00:00 Intro 01:25 The Transformer origin story 03:52 The alignment problem & attention 06:26 The architecture: encoder vs decoder 11:25 Auto-regressive LLMs & GPT 16:09 Text classification & BERT 18:51 Diffusion LLMs & LLaDA 24:17 Outro