Testing and Evaluation of Agentic AI Systems In Military Command and Control
Source
Ulysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan
Author
Ulysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan
Date
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
Established test-and-evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → methods assume a system is specifiable, stable, composable and supervisable — agentic behavior weakens all four, so a passing evaluation does not license inferences about deployed behavior.
articlePosition: Behavioral Systems Require Behavioral TestsManuel Cherep, Nikhil Singh, Pattie Maes
articleEngineering Reliable Coding Agents: Evaluating and Operating the System Around the ModelStephanie Jarmak
articleGrounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test GenerationMichele Tufano, James McClure, Jos\'e Cambronero, Runxiang Cheng, Sherry Y. Shi, Renyao Wei, Dorothy Chen, Franjo Ivan\v{c}i\'c, Livio Dalloro, Pat Rondon