Eric Jang – Building AlphaGo from scratch

Explains why AlphaGo's MCTS gives a per-move training target that avoids the credit-assignment problem naive policy-gradient RL faces across 100k-token LLM trajectories, plus a candid breakdown of which AI research tasks LLMs can already automate and which they can't. Useful mental models for anyone building or reasoning about RL and agentic research loops.

• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

• 00:00:00Basics of Go
• 00:08:17Monte Carlo Tree Search
• 00:32:04What the neural network does
• 01:00:33Self-play
• 01:25:38Alternative RL approaches
• 01:45:47Why doesn't MCTS work for LLMs
• 02:01:09Off-policy training
• 02:12:02RL is even more information inefficient than you thought
• 02:22:16Automated AI researchers

• podcastThe data black hole at the center of AI
• podcastThoughts on AI progress (Dec 2025)
• podcastSatya Nadella — How Microsoft is preparing for AGI
• podcastSatya Nadella — Microsoft’s AGI plan & quantum breakthrough

• videoWhat rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric JangDwarkesh Patel
• videoDavid Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of ExperienceSequoia Capital
• podcastAI researchers debate how close we are to recursive self-improvementDwarkesh Patel

Checking sign-in…

Loading comments…