Evaluating Agentic Code Repair Capabilities in Distributed Systems
Source
Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang, Shaoyu Wang, Huanchen Sun, Seo Jin Park
Author
Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang, Shaoyu Wang, Huanchen Sun, Seo Jin Park
Date
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
Why it matters
Coding agents cluster in the high 70s on single-process SWE-benchThe standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.Full definition →, and this shows what happens when root causes span processes and what feeding in logs and traces is worth.
articleRethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiencyJunchi Liu, Ali Bigdeli, Roya Daneshi, Atu Ambala, Sudipto Ghosh, Fabio Santos
articleWhen Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic SystemsWenji Bai, Muhammad Waseem, Zeeshan Rasheed, Jaakko Peltonen, Pekka Abrahamsson
articleAsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software EngineeringKaituo Zhang, Zhen Xiong, Zhimeng Jiang, Mingyu Zhong, Zhouyuan Yuan, Zhecheng Li, Bowen Lin, Chia-Yuan Chang, Mingzhi Hu, Huazheng Wang, Ying Lin