Vibeleaderboard
← All Intel
Intel / article

Evaluating Agentic Code Repair Capabilities in Distributed Systems

Source
arxiv.org
Author
Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang, Shaoyu Wang, Huanchen Sun, Seo Jin Park
Date

Context

A bug spread across several processes may require evidence that source code alone does not reveal. DDBench tests this by giving coding agents either a bug symptom and repository, or the same task with bounded debugging material such as logs, traces and runtime observations. Its 60 historical bugs come from 13 distributed systems. Study abstract.

Across ten models, the authors report that supplied debugging increased the aggregate repair pass rate by 18.1 percentage points. Weaker models gained successful repairs, while stronger models gained efficiency. This measures the value of providing diagnostic evidence; it leaves open how reliably an can gather that evidence itself. The authors also found that even accurate context could sometimes mislead the agent.

Terms in this piece · Glossary
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Recommended reads
Comments

Checking sign-in…

Loading comments…