When a coding agent is stuck, extra tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → are not the best lever. Task specifications improve repair most, while test feedback and retrieved code add little, yet most frontier models default to seeking feedback.
Key takeaways · AI-distilled
Banzhaf Resource Attribution splits a stuck coding AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s options into computation, information and feedback, and uses a factorial design that holds the failed state and revision protocol fixed to measure each resource's marginal contribution and interactions.
Across six models, 194 function-level failures from MBPP+ and HumanEval+ and 126 repository-level failures from SWE-benchThe standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.Full definition →, task-relevant specifications consistently improved repair; public-test feedback gave small gains and retrieved code little to none.
Extra computation helps less as model capability rises, while added information stays useful. Most resource channels show diminishing returns, though complementary information can reinforce itself.
Three of four frontier models chose feedback in 95-100% of cases and gained less than uniform random resource selection, a gap between the resources agents prefer and those that measurably improve repair.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
articleNot All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM SystemsHeming Fu, Shan Lin, Qianqian Xie, Guojun Xiong
articleSkip the Compressor: Coding Agents Fix More Issues When They Pick Whole FilesXiaocong Yang, Rishit Chatterjee, Himanshu Udupi, Vikram Ramavarapu, ChengXiang Zhai
articleWhen Evidence Changes: Evaluating Memory Repair and Re-reading in Language-Model AgentsWenhui Chu (University at Albany, State University of New York)