Richard Sutton – Father of RL thinks LLMs are a dead end
Source
Dwarkesh Patel
Author
Dwarkesh Patel
Date
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Sutton, the father of reinforcement learning, argues that no amount of LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → scaling solves on-the-job continual learning, and that a new experience-based architecture will supersede today's train-then-deploy paradigm — essential perspective for anyone betting their AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → stack on LLM scaling.
Key quotes
“Reinforcement learning is about understanding your world, whereas large language models are about mimicking people, doing what people say you should do.”
Richard Sutton
“You can’t have prior knowledge if you don’t have ground truth, because the prior knowledge is supposed to be a hint or an initial belief about what the truth is.”
Richard Sutton
“Squirrels don’t go to school. Squirrels can learn all about the world.”
Richard Sutton
“Gradient descent will not make you generalize well. It will make you solve the problem.”
Richard Sutton
“I do think succession to digital intelligence or augmented humans is inevitable.”
Richard Sutton
Chapters
00:00:00Are LLMs a dead end?
00:13:04Do humans do imitation learning?
00:23:10The Era of Experience
00:33:39Current architectures generalize poorly out of distribution
00:41:29Surprises in the AI field
00:46:41Will The Bitter Lesson still apply post AGI?
