HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals
Source
arxiv.org
Author
Jasmine Brazilek, Miles Tidmarsh, Matthias Endres, Anshuman Singh, Jeremiah Miller
Date
Context
HarvestBench tests whether an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → will spend resources to avoid harming simulated animals while completing another task. Models guide tractors through a corn harvest; an animal encounter presents a choice between driving through it or paying extra fuel to detour. The researchers score actions from game logs. Their strongest reported finding concerns instructions: five of six reasoning models killed animals in under 6% of encounters under a morality briefing, while all six exceeded 84% without it. Within this simulation, willingness to avoid harm depended heavily on the briefing, and greater model capability did not reliably predict it. Study abstract.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
articleHPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and SkillsYakup Koray Budanaz, Lorenzo Paleari, Marcin Copik, Alexandru Calotoiu, Timo Schneider, Benjamin Chen Ming Choong, Shriram Chandran, Lars Hulsbergen, Levi Widmer, Torsten Hoefler
articleSpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the SpireScott C. Lowe, Sander W. Keemink
articleMeasuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?Jundong Hu, Shekar Ramachandran