An AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → can call the correct tool and still act on the wrong order. OpenRouter illustrates this with an order identifier that passes a format check but differs from the identifier the user requested. Its testing guide separates choosing the tool, validating the argument structure, and checking the argument values. For a refund workflow, evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → may also need to verify that eligibility was checked before money was returned. These checks reveal different failures that a single measure of whether the agent called a tool would conceal.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
