Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Source
arxiv.org
Author
Michele Tufano, James McClure, Jos\'e Cambronero, Runxiang Cheng, Sherry Y. Shi, Renyao Wei, Dorothy Chen, Franjo Ivan\v{c}i\'c, Livio Dalloro, Pat Rondon
Date
Why it matters
spec-driven developmentWriting and agreeing the specification first, then having an agent implement against it — the spec, not the code, becomes the artifact you maintain.Full definition → prompting is a small change to how you brief a test-writing AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →, and this is measured evidence that the intermediate specification does the work.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
spec-driven development — Writing and agreeing the specification first, then having an agent implement against it — the spec, not the code, becomes the artifact you maintain.
articleA Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI AgentsWuyang Dai, Moses Openja, Jiho Shin, Hung Viet Pham, Song Wang
articleGroundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding AgentsJunyu Guo, Shangding Gu, Ming Jin, Javad Lavaei
articleSpecification-first convergence with an AI coding agent: a case study of dismantling a core architectural invariant across 189 files in a 717k-line codebase with no test oracle and no human code reviewJoel Abenhaim