Intel / video

Verifiable Environments for AI in Biology — Kenny Workman, LatchBio

Source: youtube.com
Author: AI Engineer
Date: Published Jul 31, 2026

Context

A biology can mark a scientifically reasonable analysis wrong if its grading rules assume only one valid method. LatchBio's presentation describes encountering this while developing SpatialBench, which tests agents on experimental data analysis. Scientists reviewing the tasks found ambiguities in instructions and numerical thresholds that did not adequately account for alternative analysis choices. The team consequently produced a human-verified subset. Its proposed standard is to check conclusions or properties that remain valid across acceptable methods, while requiring the to work with the supplied data. That helps separate failures of biological reasoning from disagreements created by an underspecified question or overly narrow grader.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.

Key quotes

“the output of a single experiment can exceed what a scientist can safely store on a consumer laptop in many cases”
Kenny Workman

“just like code provided a verifiable substrate for complex software tasks that are not inherently verifiable, uh data analysis might do the same thing in bio”
Kenny Workman

“frontier models cannot be trusted to do real work. They're missing some capability between knowing biology and writing code.”
Kenny Workman

“cool thing about evaluation like coding is it forces you to reason about things more rigorously than you would when you're doing the thing yourself”
Kenny Workman

“We try to get the labs to compete um on the benchmarks cuz then it makes the models better at our products.”
Kenny Workman

Read the source www.youtube.com