AgentGuard: Learning Execution Guardrails from Anomalous Coding-Agent Trajectories
Source
arxiv.org
Author
Wuyang Dai, Song Wang
Date
Context
AgentGuard turns recurring coding-AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → failures into instructions intended to prevent their repetition. It extracts patterns from execution records, such as changing unrelated files or ignoring failed checks, and activates constraints relevant to the current instruction. In a held-out evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → covering 100 repository tasks, using Anthropic's Claude Code with Haiku 4.5, the authors report that abnormal execution fell from 69.0% to 26.7%, while successful completion rose from 21.7% to 35.0%. The reported result suggests that targeted behavioral guidance can improve both measures in this setting, although most tasks still were not completed successfully.
Key takeaways · AI-distilled
AgentGuard automatically mines recurring failure patterns from past coding-agent trajectories and turns them into instruction-level constraints that only activate when relevant, rather than relying on hand-written safety rules.
guardrailsThe checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.Full definition → were learned from 461 failure traces covering 282 repository tasks and then tested on a disjoint set of 100 tasks to check generalization.
In the authors' test of Claude Code with Claude Haiku 4.5, AgentGuard cut abnormal execution from 69.0% to 26.7% and raised successful task completion from 21.7% to 35.0%. These are results on the study's task set.
The authors note a remaining tradeoff between enforcing safety constraints and preserving task completion, meaning guardrails still leave room for missed or overly cautious behavior.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
guardrails — The checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.
articleOpenAgentFlow: Enabling System-Wide Safety Boundaries for Heterogeneous AI Agent FleetsDongsheng Chen, Xiangyu Zhao, Xin Yao, Xuetao Wei
articleTeaching Agents to Code ReliablyMuhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan
articleHumanize: Judgement Engineering for Agentic CodingSihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki