The next big breakthrough will be AIs learning on the job
Source
Dwarkesh Patel
Author
Dwarkesh Patel
Date
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
A clear-eyed argument for why continual learning — getting model improvements back into the weights rather than just context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → — is the bet frontier labs are making, with concrete framing (RLVR generalization, verifiability vs grindability) that helps engineers reason about where AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → capabilities are headed by 2027.
Key quotes
“You can’t have a thousand agents go try the same checkout flow on Amazon.com. Because Andy Jassy will find and detect your bots and shut your ass down.”
Dwarkesh Patel
“We’ve got some genius grad student who has never been allowed to take an internship.”
Dwarkesh Patel
“Around 30-50% of a lab’s compute goes to inference, and that compute is currently not really doing anything productive in helping improve the model. What a waste!”
Dwarkesh Patel
“the way you get better at your job is not by recalling the transcript of what happened through every single day with perfect fidelity”
Dwarkesh Patel
“I just don’t think you can accumulate new skills by passing yourself notes.”
Dwarkesh Patel
Chapters
00:00:00The big research bet the labs are making
00:02:12Grindability is just as important as verifiability
