Clip / AI Agents

Forking Real Deployments to Create Low-Awareness Simulations

From Vending-Bench: Long-Horizon Agent Evals — Lukas Petersson, Andon Labs · ≈14:01

“So, we take the real life environments, we fork them so that like up until the fork, the agent is in the real world, but after the fork, it's in simulation.”

“And we've seen that like this dramatically decreases simulation awareness.”

What’s in it

Explains a trick to fork real environments into simulations for agents
Reveals why agents often detect they're being tested in sim
Shows how early-turn cloning fools models into acting naturally

Watch this moment

Clip transcript

some we can some way we can get the best of both worlds? Maybe. Um, we hope yes. Uh, we've experimented a lot lately with like making digital clones of the of the environment. So, we take the real life environments, we fork them so that like up until the fork, the agent is in the real world, but after the fork, it's in simulation. And we've seen that like this dramatically decreases simulation awareness. Like the first few like turns, it's like absolutely impossible for the model to know that it's in the