Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model
- Source
- arxiv.org
- Author
- Stephanie Jarmak
- Date

Context
A coding 's failure can originate outside the model that writes its code. This engineering monograph examines the surrounding system, including execution environments, information retrieval, retained state, permissions and verification. Its central argument is that weaknesses in these components can invalidate conclusions drawn from a model's apparent performance.
The work organizes evidence from research, practitioner accounts, benchmarks and operated systems into a reliability framework. It gives teams a way to investigate where a failure begins and whether a local improvement reaches the completed task. As a structured review, its recommendations have differing evidence strength and depend on the workload and configuration.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
articleTeaching Agents to Code ReliablyMuhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan
articleCompound AI System Reliability: A Failure Taxonomy and Resilience Pattern Catalog from 150 Production IncidentsRudrendu Kumar Paul, Sourav Nandy
articleFrom Traceability to Justifiability: Accountability Structures in Agentic Software EngineeringRashid Azarang
Checking sign-in…
Loading comments…