Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Source
arxiv.org
Author
Harsh Raj, David Lee, Anas Mahmoud, Renxiong Wang, Razvan-Gabriel Dumitru, Chenguang Wang, Tong Zhao, Yunzhong He, Darvin Yi, Vipul Gupta
Date
Context
An AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s visible failure may occur far from the action that caused it. Continual Search addresses this by repeatedly prompting a diagnostic model to challenge its explanation and inspect evidence it has not yet examined, such as unread tool outputs. The comparison method merely asks it to reconsider its existing answer.
On MegaRCA-Mix, a collection of 50 human-annotated failures, OpenAI's GPT-5.5 improved from 0.349 to 0.498 on the study's F1 attribution metric; simple reconsideration reached 0.401. The benefit was strongest where lengthy records left evidence unexplored. Shorter-record benchmarks did not improve consistently, making this a targeted debugging technique for large execution histories.
Key takeaways · AI-distilled
The paper reframes diagnosing why an AI agent failed as a search problem: existing one-shot LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →-judge methods tend to lock onto a plausible-looking diagnosis early and leave critical evidence buried in long execution traces unexamined.
Its proposed method, Continual Search, repeatedly nudges the judge across multiple turns to keep hunting for unresolved diagnostic evidence instead of stopping at the first plausible answer.
On a new 50-trial long-horizon benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → built specifically because existing ones lack sufficiently large traces, Continual Search lifts GPT-5.5's diagnostic F1 score from 0.349 to 0.498, over a 40% improvement.
Within the same model family, a lower-tier model using Continual Search can outperform a higher-tier model that doesn't use it, suggesting better search can substitute for raw model scale on this task.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
articleKRCA: An Efficient Root Cause Analysis System in Hyper-Scale Microservice Systems via Agentic AIJiamin Jiang, Jingfei Feng, Yu Luo, Qingliang Zhang, Yongqian Su, Wenwei Gu, Shenglin Zhang, Tianyu Cui, Yao Wu, Jielong Huang, Nan Qi, Dan Pei
articleEviRCA: Decoupling Evidence Extraction from Reasoning for Microservice Root-Cause AnalysisYuhao Wang, Zhen Qin, Xingliang Wang, Guochang Li, Weize Li, Shuiguang Deng
articleDoes Graph Structure Earn Its Place in Microservice Root-Cause Analysis? A Controlled Study on RCAEval, and What the Benchmark Was Really MeasuringImad Bulji\'c