The Breakthroughs Needed for AGI Have Already Been Made: OpenAI Former Research Head Bob McGrew
- Source
- youtube.com
- Author
- Sequoia Capital
- Date
Context
Bob McGrew, a former chief research officer at OpenAI, argues in a Sequoia Training Data interview that the basic ideas needed for ever more capable AI are already known: language models built on transformers, scaling up (the large first run on text), and reasoning (training models to work through a problem before answering), with abilities woven through. His evidence is his recollection of 2020, after GPT-3, when he says the internal roadmap already pointed to bigger pre-training, multimodality and , and that computer use took years only because multimodality needed time to mature.
McGrew expects remaining progress to be hard refinement and scaling rather than new concepts, and says pre-training faces diminishing returns because each increment of intelligence needs exponentially more compute and a new data center. He calls the claim potentially controversial and says he hopes he is wrong. It is a prediction, with 2035 as its test.
- Training — The process that adjusts a model using examples and a measure of how well it performs.
- pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
- multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
- test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
videoBringing Continual Learning to Every Company: Trajectory’s Ronak Malde
videoCrusoe CEO Chase Lochmiller: Turning Stranded Energy Into Intelligence
videoDavid Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience
videoCracking the Code on Offensive Security With AI ft XBOW CEO and GitHub Copilot Creator Oege de Moor
Checking sign-in…
Loading comments…


