Gives you precise, shared terminology for building agent evaluation harnesses and explains specifically why agentic evals fail differently than single-turn evals (compounding errors, valid-but-unexpected solutions), which helps you design graders and suites that don't break on production agents.
An Anthropic engineering post that lays out a vocabulary and framework for evaluating AI agents—covering tasks, trials, graders, transcripts, outcomes, evaluation harnesses, and evaluation suites—and explains why agentic evals are harder than single-turn evals because mistakes can compound across tool calls and models can find valid solutions that 'fail' rigid grading criteria.
It argues that building rigorous evals early prevents teams from getting stuck fixing issues reactively once an agent is in production.
