Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software

An agent's activity log may be insufficient to establish that its work succeeded. Theta Software's presentation uses a deployment-repair task to explain the difference: editing code and requesting another deployment leave a record of actions, but the evaluator still needs to check whether the service works afterward. The proposed evaluator can inspect the resulting environment and search the agent's history for relevant decisions. Its access should be read-only so that verification does not change the outcome being judged. This approach makes the final condition of the system part of the evidence, which matters when several different sequences of actions could legitimately solve the task.

“what we consider long horizon a year ago probably isn't really long horizon in our definition today”

“one task can you know maybe be made by artificially long horizon by chaining together unrelated independent tasks”

“I think the first important uh consideration to make is that judges are agents too”

“if we kind of enforce this too tightly, we collapse the state space of how many actual paths the agent actually explores”

“a lot of the literature shows that a lot of the data being produced right now and being used to train and evaluate models is actually flawed”

Rayan Garg