What Makes Software Issue Resolution Tasks Difficult for Agents?
Source
arxiv.org
Author
Ebtesam Al-Haque, Brittany Johnson
Date
Why it matters
Makes AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → numbers readable by attaching difficulty to the tasks behind them.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleXAgent: eXecution-guided Agentic AI for Effective Localization and Resolution of GitHub IssuesHieu Huynh, Patanamon Thongtanunam, Michael Fu, Bach Le, Kla Tantithamthavorn
articleAsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software EngineeringKaituo Zhang, Zhen Xiong, Zhimeng Jiang, Mingyu Zhong, Zhouyuan Yuan, Zhecheng Li, Bowen Lin, Chia-Yuan Chang, Mingzhi Hu, Huazheng Wang, Ying Lin
articleFinishing the Task Is Not Enough: Evaluating Agent Resilience and Considerate Participation under Accumulating ChallengeYuanchen Bai, Zijian Ding, Angelique Taylor