π¬ Google's AI Scientist Started as an Attempt to Automate Kaggle β John Platt, Google Fellow
- Source
- youtube.com
- Author
- Latent Space
- Date
Context
Google's Empirical Research Assistance, or ERA, searches for better scientific software once researchers can say how to score it. A researcher supplies a problem description, a scoring metric, and data for training, validation, and . ERA then generates ideas, implements each as executable code, tests it in a against the score, and uses a tree search with an upper confidence bound to decide which candidates deserve further work, with a rewriting code to try to raise the score. The output is code that researchers can inspect and rerun. Google Research explanation
The hard part stays with people. In the Latent Space interview with Google Fellow John Platt, the episode description frames choosing the score as the hard part and notes that predictive accuracy does not automatically mean scientific understanding. A higher score shows the code predicts better under that metric, so researchers still have to confirm the metric rewards the outcome they want. Interview description
- Platt credits ERA with helping crack a climate-modeling problem that had stalled for two years, and explains that contrails trap heat and that small changes in flight altitude can reduce that warming.
- Platt still recommends starting with a simple baseline such as linear regression or an SVM, and cautions that predictive accuracy does not automatically mean scientific understanding.
- The episode uses a half-pixel labeling error that helped win a Kaggle competition to illustrate reward hacking and Goodhart's law, a risk for any optimizing a chosen score.
- Google's FireSat effort aims to detect wildfires from space before they grow out of control, one of the climate-resilience applications Platt discusses.
- eval β A repeatable test for AI quality β a set of tasks plus scoring β used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- sandbox β An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
- LLM β A large language model β the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
- AI agent β An AI system that doesn't just answer once but works toward a goal in a loop β taking actions, reading the results, and deciding what to do next.
article㪠An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for ScienceBrandon Anderson
podcastTerence Tao β Kepler, Newton, and the true nature of mathematical discoveryDwarkesh Patel
podcastAI researchers debate how close we are to recursive self-improvementDwarkesh Patel
Checking sign-inβ¦
Loading commentsβ¦



