Richard Sutton – Father of RL thinks LLMs are a dead end
- Source
- dwarkesh.com
- Author
- Dwarkesh Patel
- Date
Sutton, the father of reinforcement learning, argues that no amount of scaling solves on-the-job continual learning, and that a new experience-based architecture will supersede today's train-then-deploy paradigm — essential perspective for anyone betting their stack on LLM scaling.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
“Reinforcement learning is about understanding your world, whereas large language models are about mimicking people, doing what people say you should do.”
Richard Sutton
“You can’t have prior knowledge if you don’t have ground truth, because the prior knowledge is supposed to be a hint or an initial belief about what the truth is.”
Richard Sutton
“Squirrels don’t go to school. Squirrels can learn all about the world.”
Richard Sutton
“Gradient descent will not make you generalize well. It will make you solve the problem.”
Richard Sutton
“I do think succession to digital intelligence or augmented humans is inevitable.”
Richard Sutton
- 00:00:00Are LLMs a dead end?
- 00:13:04Do humans do imitation learning?
- 00:23:10The Era of Experience
- 00:33:39Current architectures generalize poorly out of distribution
- 00:41:29Surprises in the AI field
- 00:46:41Will The Bitter Lesson still apply post AGI?
- 00:53:48Succession to AIs
Checking sign-in…
Loading comments…