Adaption Labs: Gradient-Free Continual Learning — Sara Hooker, Adaption

AutoScientist is presented as a way to automate choices across model development, including training data and model design. The conference description says it searches across model sizes and architectures, and that the approach became useful only after data quality was controlled alongside the model.

The proposed benefit is a broader search than researchers might attempt when relying on familiar designs. But the reported win rates, slightly above 60%, came from experiments with a budget-based stopping rule. They describe those comparisons, not a general probability that an automated system will outperform a research team.

“Fewer than five thousand people in the world know how to train a frontier model at scale, by Sara Hooker's estimate, and that knowledge travels like an apprenticeship rather than a literature.”

AI Engineer

“Modern computer science is 77 years old, two generations, and in that time the route to contributing at the frontier narrowed into one funnel: the right PhD, the right industry lab, the right problem at the right moment.”

AI Engineer

“AutoScientist automates the training of models, optimizing the whole loop together from data through alignment and evolving itself per domain, and it beats research staff partly because people carry priors about particular architectures while the search ranges across sizes and across dense and mixture of experts designs.”

AI Engineer

“One nice piece of honesty: the win rates all sit just above 60% because the budget was set to stop there, and lifting that ceiling let them keep climbing.”

AI Engineer

“That matters for access, because pretraining compute has to be colocated and enormous while the compute that now pays off is distributable. If no lab is going to quadruple model size again on this architecture, recipes and algorithms start to matter more than hoarded GPUs.”

AI Engineer