Assurance Envelopes for Autonomous Coding Agents: Minimum-Cost Evidence for Software Change
Source
arxiv.org
Author
Anjan Goswami
Date
Context
An assurance envelope is a proposed way to select the evidence a coding AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → needs before changing existing software. Starting with specified properties the change must preserve, the method connects prior tests, analyses and other evidence through rules describing what they jointly establish. It chooses a least-cost subset, then checks that the selected evidence actually supports every required property. The study demonstrates this selection problem on small software-derived graphs and synthetic cases. Its scope begins after the requirements are known: discovering those requirements and showing that the selection improves later agent work remain open.
Key takeaways · AI-distilled
Anjan Goswami frames reusing a coding agent's prior evidence (tests, type checks, proofs, analyses, traces) as selecting a least-cost 'task-conditioned assurance envelope' that must fully re-establish the properties a code change needs to preserve.
Tested on small evidence graphs from Rust, IronBlocks, and Pong projects, the minimum needed evidence set varies by task, and sometimes no valid subset exists if current evidence can't reestablish a required property.
In the studied evidence graphs, some required properties needed several pieces of evidence together. Expanding a task's requirements added needed evidence rather than replacing the existing set.
On a 249-instance synthetic benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →, every completed exact cross-check matched the study's constraint-solving optimizer. Median solve time stayed under 20 milliseconds for graphs with 500 evidence items.
Graphs with many alternative ways to establish the same result timed out at much smaller sizes. In this benchmark, how evidence was connected mattered more for solving difficulty than the number of evidence items.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
articleKnowing When to Stop: The Evidence-Action Gap in Coding AgentsTianyu Chen, Xinran Zhao, Vijay Viswanathan, Shoubin Yu, Mingyuan Zhou, Tongshuang Wu
articleWhen Evidence Changes: Evaluating Memory Repair and Re-reading in Language-Model AgentsWenhui Chu (University at Albany, State University of New York)
articleTeaching Agents to Code ReliablyMuhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan