Compound AI System Reliability: A Failure Taxonomy and Resilience Pattern Catalog from 150 Production Incidents
- Source
- arxiv.org
- Author
- Rudrendu Kumar Paul, Sourav Nandy
- Date

Context
Many failures in systems built from several AI components happen at the boundaries between components, not inside any one model, according to a paper accepted at an ICML 2026 workshop. The authors analyzed 150 incident reports from open-source projects and anonymized enterprise deployments and sorted them into 23 failure modes in five categories: retrieval, generation, tool, orchestration and integration.
For each category they propose resilience patterns and report results from controlled fault injection, where faults are deliberately introduced. Circuit breakers, which stop a failing part from passing errors onward, cut cascade propagation by 89%, output quality gates caught 73% of silent degradation before user impact, and component isolation reduced blast radius by 64%. Systems using three or more patterns reduced mean time to recovery by 71% against unstructured monitoring.
The abstract does not say how incidents were selected or how many systems were compared, so these are the authors' reported results, not established general effects. For related work on injecting faults into systems, see this earlier item on AgentChaos.
- The authors built their taxonomy from 150 production incident reports drawn from open-source compound AI projects and anonymized enterprise deployments, yielding 23 failure modes in five categories: retrieval, generation, tool, orchestration and integration failures.
- Their core argument is that compound AI systems mostly break at component boundaries rather than inside a single model: errors cascade between components, quality degrades silently past standard monitoring, and correct parts can still coordinate into wrong behavior.
- In the authors' controlled fault-injection experiments, circuit breakers cut cascade propagation by 89%, output quality gates caught 73% of silent degradation before users saw it, and component isolation shrank blast radius by 64%.
- The paper reports that systems using three or more patterns from its catalog cut mean time to recovery by 71% compared with unstructured monitoring baselines.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleEngineering Reliable Coding Agents: Evaluating and Operating the System Around the ModelStephanie Jarmak
articleAIFaultBench: A Reproducible Benchmark of Real-World Faults in AI Software SystemsMehil B Shah, Mohammad Masudur Rahman, Foutse Khomh
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li
Checking sign-in…
Loading comments…