Why sample efficiency — not compute — may be the real bottleneck in AI progress.
Key quotes
“We see these AIs as a galaxy glittering with capabilities, but at their center, invisible to the naked eye, holding all the constellations together, is an unimaginably massive black hole of data.”
Dwarkesh Patel
“We are building some Frankenstein’s monster, with a billion grafts of carefully constructed examples sewn together.”
Dwarkesh Patel
“Humans are somewhere between thousands to millions of times more sample efficient than these models. Scaling of current models simply can’t make up for that discrepancy.”
Dwarkesh Patel
“Imagine if it took a couple decades worth of courses with hundreds of concurrent professors and millions of practice tasks for you to learn how to polish a word file.”
Dwarkesh Patel
“I would be willing to bet that there’s overall more demand for human software engineers in 2028 than there is now, largely due to the complementary input of AI.”
