Ilya Sutskever — We're moving from the age of scaling to the age of research
Source
Dwarkesh Patel
Author
Dwarkesh Patel
Date
Terms in this piece · Glossary
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
Why it matters
Ilya Sutskever lays out why the field is shifting from scaling to research — covering model jaggedness, generalization gaps, and how SSI plans to reach superintelligence — giving engineers a sharper mental model of where frontier capabilities are actually bottlenecked.
Key quotes
“So it’s back to the age of research again, just with big computers.”
Ilya Sutskever
“these models somehow just generalize dramatically worse than people. It’s super obvious.”
Ilya Sutskever
“We got to the point where we are in a world where there are more companies than ideas by quite a bit.”
Ilya Sutskever
“In theory, there is no difference between theory and practice. In practice, there is.”
Ilya Sutskever
“The top-down belief is the thing that sustains you when the experiments contradict you.”
Ilya Sutskever
Chapters
00:00:00Explaining model jaggedness
00:09:39Emotions and value functions
00:18:49What are we scaling?
00:25:13Why humans generalize better than models
00:35:45SSI's plan to straight-shot superintelligence
00:46:47SSI's model will learn from deployment
00:55:07How to think about powerful AGIs
01:18:13We are squarely an age of research company
