Demis Hassabis: Agents, AGI & The Next Big Scientific Breakthrough
- Source
- youtube.com
- Author
- Y Combinator
- Date
Context
Demis Hassabis, who leads Google DeepMind, says in a Y Combinator talk that large-scale , and reasoning will likely be part of artificial general intelligence (AGI), but that pieces remain unsolved: continual learning (taking in new knowledge without retraining), long-term reasoning, parts of memory and consistency across tasks. Hassabis puts the odds about even on whether existing techniques plus incremental innovation suffice or one or two big ideas are missing, and gives a personal timeline of about 2030.
On memory, Hassabis calls stuffing everything into a long brute force. A window works like working memory, but it fills with unimportant or wrong material, and finding the relevant piece has a cost even when capacity is huge. As an illustration, a million tokens covers only about 20 minutes of live video if all tokens are recorded naively, so an agent following someone's life for weeks needs something better. These are opinions from the head of a leading lab, not measured results.
- pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
- RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
- chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
videoDemis Hassabis: Why AGI is Bigger than the Industrial Revolution & Where Are The Bottlenecks in AI20VC
podcast#475 – Demis Hassabis: Future of AI, Simulating Reality, Physics and Video GamesLex Fridman
videoThe Breakthroughs Needed for AGI Have Already Been Made: OpenAI Former Research Head Bob McGrewSequoia Capital
Checking sign-in…
Loading comments…



