Adaption Labs: Gradient-Free Continual Learning — Sara Hooker, Adaption
- Source
- AI Engineer
- Author
- AI Engineer
- Date
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
A case, with system results behind it, that the axis of progress is shifting away from scale and toward automated search over the training loop.
“Fewer than five thousand people in the world know how to train a frontier model at scale, by Sara Hooker's estimate, and that knowledge travels like an apprenticeship rather than a literature.”
AI Engineer
“Modern computer science is 77 years old, two generations, and in that time the route to contributing at the frontier narrowed into one funnel: the right PhD, the right industry lab, the right problem at the right moment.”
AI Engineer
“AutoScientist automates the training of models, optimizing the whole loop together from data through alignment and evolving itself per domain, and it beats research staff partly because people carry priors about particular architectures while the search ranges across sizes and across dense and mixture of experts designs.”
AI Engineer
“One nice piece of honesty: the win rates all sit just above 60% because the budget was set to stop there, and lifting that ceiling let them keep climbing.”
AI Engineer
“That matters for access, because pretraining compute has to be colocated and enormous while the compute that now pays off is distributable. If no lab is going to quadruple model size again on this architecture, recipes and algorithms start to matter more than hoarded GPUs.”
AI Engineer
videoDistill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash
videoTeaching LLMs to Speak Spotify — Yves Raimond & Jacqueline Wood, Spotify
videoWhy LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, Meta
videoWorld Models Need Causality, Not Pretty Pixels — Christopher Manning, Moonlake AI
Checking sign-in…
Loading comments…