Vibeleaderboard
← All Intel
Intel / video

πŸ”¬ Google's AI Scientist Started as an Attempt to Automate Kaggle β€” John Platt, Google Fellow

Source
youtube.com
Author
Latent Space
Date

Context

Google's Empirical Research Assistance, or ERA, searches for better scientific software once researchers can say how to score it. A researcher supplies a problem description, a scoring metric, and data for training, validation, and . ERA then generates ideas, implements each as executable code, tests it in a against the score, and uses a tree search with an upper confidence bound to decide which candidates deserve further work, with a rewriting code to try to raise the score. The output is code that researchers can inspect and rerun. Google Research explanation

The hard part stays with people. In the Latent Space interview with Google Fellow John Platt, the episode description frames choosing the score as the hard part and notes that predictive accuracy does not automatically mean scientific understanding. A higher score shows the code predicts better under that metric, so researchers still have to confirm the metric rewards the outcome they want. Interview description

Key takeaways Β· AI-distilled
  • Platt credits ERA with helping crack a climate-modeling problem that had stalled for two years, and explains that contrails trap heat and that small changes in flight altitude can reduce that warming.
  • Platt still recommends starting with a simple baseline such as linear regression or an SVM, and cautions that predictive accuracy does not automatically mean scientific understanding.
  • The episode uses a half-pixel labeling error that helped win a Kaggle competition to illustrate reward hacking and Goodhart's law, a risk for any optimizing a chosen score.
  • Google's FireSat effort aims to detect wildfires from space before they grow out of control, one of the climate-resilience applications Platt discusses.
Terms in this piece Β· Glossary
  • eval β€” A repeatable test for AI quality β€” a set of tasks plus scoring β€” used the way software teams use test suites, because model output is too variable to judge by eyeballing.
  • sandbox β€” An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
  • LLM β€” A large language model β€” the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • AI agent β€” An AI system that doesn't just answer once but works toward a goal in a loop β€” taking actions, reading the results, and deciding what to do next.
Read the source www.youtube.com
More from Latent Space
Recommended reads
Comments

Checking sign-in…

Loading comments…