Richard Sutton – Father of RL thinks LLMs are a dead end
Executive Summary
Richard Sutton and Dwarkesh Patel disagree about whether language models are a good foundation for intelligence that learns during use. This summary covers the technical debate, experiential learning, generalization and later knowledge-sharing risks, not the closing succession discussion. Sutton's critiques are his September 2025 views, challenged by Patel rather than treated as settled findings.
Key Takeaways
Prediction of words is not the same question as consequences
Sutton distinguishes predicting what people say from predicting what happens after an action. Patel counters that language models appear to contain useful world models. Their disagreement turns on what counts as prediction and whether unexpected outcomes cause durable changes, not simply whether outputs sound knowledgeable.
A goal makes better and worse testable
For Sutton, intelligence requires achieving goals, and reward provides a way to assess actions during ordinary interaction. Patel points to math-solving systems with explicit objectives. Sutton accepts the distinction between proof search and learning empirical consequences while disputing how far the former establishes the latter.
Continual learning is the central missing capability
Sutton wants systems to learn from the stream of action and observation throughout their lives. Patel asks why an LLM could not provide the starting point. The interview does not resolve that architecture question, so the title's dead-end claim should be read as Sutton's position, not a demonstrated impossibility.
The animal analogy is contested
Sutton emphasizes trial-and-error and prediction in animals, while Patel stresses imitation and cultural knowledge in human learning. Sutton later agrees that imitation can transmit culture but views it as built on more basic learning. Their contrasting emphasis explains part of their disagreement without proving a complete theory of human cognition.
Long goals can have shorter feedback loops
Sutton explains temporal-difference learning with chess: a value estimate changes when a move improves the expected eventual outcome, helping reinforce steps before the game ends. He applies the same reasoning to long projects. The example clarifies credit assignment; it is not a promise that a chosen proxy accurately captures the final goal.
An experiential agent needs more than reward
He describes policy, value function, perception and a transition model. The transition model learns richly from observations of what actions lead to, rather than only from a reward number. This answers Patel's question about how an agent could absorb the large amount of local information needed on a job.
Generalization and knowledge sharing need tests
Sutton challenges whether current methods promote good transfer rather than merely solve seen problems; Patel offers coding improvements as a counterexample. Later Sutton raises the risk that merging knowledge from digital copies could also import harmful changes. Both discussions favor testing outcomes and assumptions instead of assuming transfer or merging is safe.
The Conversation Breakdown
What the technical disagreement turns on
⏱ 00:00Sutton favors learning from experience with goals and observable consequences. Patel probes whether existing language models can become that foundation. The discussion moves through animal learning, temporal-difference feedback, agent components and transfer, then asks how digital intelligences might share knowledge. This is a selected technical summary; the closing succession debate is not summarized here.
Core Frameworks & Ideas
Experience loop
Observe a situation, choose an action, see the consequence and update. Sutton treats knowledge as testable claims about that stream.
Four-part agent
Policy chooses actions; value estimates how things are going; perception represents the current state; a transition model predicts consequences. Reward is only part of the learning signal.
Temporal-difference feedback
Use changes in predicted long-term value to learn from intermediate steps, while checking that those predictions remain tied to the actual goal.
Actionable Next Steps
- ✓For an AI evaluation, define the goal and the observable consequence separately from output quality.
- ✓Ask whether a system retains lessons across interactions or only adapts within the current context.
- ✓Test performance on new states and check whether learning new material damages older skills.
- ✓Before importing learned information into a system, test for unwanted behavior changes and preserve a rollback route.
Notable Quotes
“The scalable method is you learn from experience. You try things, you see what works.”
“We don’t just learn from the reward. We learn from all the data.”
“I really view myself as a classicist rather than as a contrarian.”
The One Thing to Remember
Define what feedback changes a system during use before calling it a continual learner.