Position: Behavioral Systems Require Behavioral Tests
Source
arxiv.org
Author
Manuel Cherep, Nikhil Singh, Pattie Maes
Date
Why it matters
Argues for a class of AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → that current outcome only harnesses do not cover.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleAI Evaluation Should Work With HumansJan Kulveit, Gavin Leech, Tom\'a\v{s} Gaven\v{c}iak, Raymond Douglas
articleTesting and Evaluation of Agentic AI Systems In Military Command and ControlUlysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan
articlePosition: Multi-Agent Systems Should Prioritize Concurrency ControlXin Yang, Letian Li, Zimo Ji, Terry Jingchen Zhang, Wenyuan Jiang