Vibeleaderboard
← All Intel
Intel / video

The Breakthroughs Needed for AGI Have Already Been Made: OpenAI Former Research Head Bob McGrew

Source
youtube.com
Author
Sequoia Capital
Date

Context

Bob McGrew, a former chief research officer at OpenAI, argues in a Sequoia Training Data interview that the basic ideas needed for ever more capable AI are already known: language models built on transformers, scaling up (the large first run on text), and reasoning (training models to work through a problem before answering), with abilities woven through. His evidence is his recollection of 2020, after GPT-3, when he says the internal roadmap already pointed to bigger pre-training, multimodality and , and that computer use took years only because multimodality needed time to mature.

McGrew expects remaining progress to be hard refinement and scaling rather than new concepts, and says pre-training faces diminishing returns because each increment of intelligence needs exponentially more compute and a new data center. He calls the claim potentially controversial and says he hopes he is wrong. It is a prediction, with 2035 as its test.

Terms in this piece · Glossary
  • Training — The process that adjusts a model using examples and a measure of how well it performs.
  • pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
  • multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
  • test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
Read the source www.youtube.com
More from Sequoia Capital
Recommended reads
Comments

Checking sign-in…

Loading comments…