Checking that an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → called the right tool does not establish that it explained the result correctly. OpenRouter'sevaluation tutorial adds a second model to grade the final answer against explicit criteria, such as whether it states the retrieved refund window without inventing exceptions. The tutorial pairs these judgments with ordinary checks on tool execution and tests the grader against human-labeled examples. That combination evaluates both what the agent did and whether its response accurately represents the evidence.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
