Intel / article

🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

Source: latent.space
Author: Brandon Anderson
Date: Published Sep 22, 2026

Context

Google's Empirical Research Assistance (ERA) system starts with a scientific problem that can be scored. A researcher supplies the problem, data and rule; the system generates code, tests alternatives and uses their results to choose which approaches to explore further. Google's technical account describes this as a tree search over candidate programs. It presents the work as a research paper with shared top solutions, and the interview write-up below notes ERA is not currently a Google product.

The central caution in researcher John Platt's interview write-up is that the score is only a stand-in for the science. The write-up reports Platt's view that ERA produces predictive models and that scientists must make sure they are truly descriptive of the phenomenon. It illustrates the risk with Goodhart's law: a metric that becomes a target stops being a good measure, as when contest entrants exploited a labeling error. Automating the search for better-scoring code leaves that judgment with the scientist.

Key takeaways · AI-distilled

ERA (Google's Empirical Research Assistance) treats scientific problems as scoreable tasks: an keeps a tree of past experiment notebooks and picks which to mutate via an Upper Confidence Bound rule, proposing about ten mutations per iteration.
Platt says ERA only started working well after a step change between Gemini 2.0 and 2.5, going from not functioning to functioning great, since this kind of evolutionary search needs the underlying model to actually know where to look.
Using ERA, Platt's team found contrails, the ice-crystal trails from jet exhaust, account for about 1% of human-induced global warming; a single gram of engine exhaust can seed roughly ten kilograms of ice crystals.
The fix for contrails, flying a level or two lower through ice-supersaturated air, was already known, but modeling how much warming it actually prevented stumped Platt's team for two years until ERA found a simpler model accounting for reflected sunlight they had missed.
Platt warns ERA is a power tool that can slice your fingers off: in a Google contrail-detection Kaggle competition, winning entrants exploited a half-pixel labeling error rather than solving the underlying problem, illustrating Goodhart's law in practice.

Terms in this piece · Glossary

eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source www.latent.space