Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Source
Michele Tufano, James McClure, Jos\'e Cambronero, Runxiang Cheng, Sherry Y. Shi, Renyao Wei, Dorothy Chen, Franjo Ivan\v{c}i\'c, Livio Dalloro, Pat Rondon
Author
Michele Tufano, James McClure, Jos\'e Cambronero, Runxiang Cheng, Sherry Y. Shi, Renyao Wei, Dorothy Chen, Franjo Ivan\v{c}i\'c, Livio Dalloro, Pat Rondon
Published
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
spec-driven development — Writing and agreeing the specification first, then having an agent implement against it — the spec, not the code, becomes the artifact you maintain.
Why it matters
spec-driven developmentWriting and agreeing the specification first, then having an agent implement against it — the spec, not the code, becomes the artifact you maintain.Full definition → prompting is a small change to how you brief a test-writing AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →, and this is measured evidence that the intermediate specification does the work.
articleSpecification-first convergence with an AI coding agent: a case study of dismantling a core architectural invariant across 189 files in a 717k-line codebase with no test oracle and no human code reviewJoel Abenhaim
articleSpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from ScratchYihao Chen, Shi Chang, Feng Lin, Khaled Chawa, Boyuan Chen, Shaowei Wang, Ahmed E. Hassan
articleHxAgent: Iterative Agent Planning for End-to-End Web Application TestingTu Nguyen, Duy Cao, Viet Nguyen, Phu Nguyen, Vy Le, Nguyen TK Nguyen, Tien N. Nguyen, Vu Nguyen