Blocks coding agents from editing code before verifying enough evidence
Raises SWE-bench Verified Pass@1 by 4.8-11.8 points, no retraining needed
Cuts token usage up to 12.1% by preventing unsupported agent actions
Key quotes
“LLM-based coding agents often edit source code or submit patches before examining enough repository evidence to justify the change, a failure pattern we call premature commitment.”
“Evaluated on all 500 instances of SWE-bench Verified with two language models and two agent scaffolds, ECLoop raises Pass@1 by 4.8-11.8 percentage points without model retraining or scaffold changes.”
“These gains come at no additional inference cost: by redirecting the agent before it pursues unsupported actions, ECLoop lowers average token consumption by up to 12.1%.”
Transcript
arXiv:2607.28815v1 Announce Type: new Abstract: LLM-based coding agents often edit source code or submit patches before examining enough repository evidence to justify the change, a failure pattern we call premature commitment. We present ECLoop, an execution layer that interposes between the agent and the repository to enforce evidence-conditioned execution. For each task, ECLoop uses the issue description and repository structure to compile a set of conditions specifying what the agent should observe before each type of code modification or patch submission. During execution, ECLoop tracks which conditions the agent's runtime trajectory has satisfied and postpones any proposed action whose required conditions remain unmet. Evaluated on all 500 instances of SWE-bench Verified with two language models and two agent scaffolds, ECLoop raises Pass@1 by 4.8-11.8 percentage points without model retraining or scaffold changes. Ablation experiments show that each of ECLoop's three operations contributes distinct value and that structured evidence conditions outperform an equivalent natural-language summary. These gains come at no additional inference cost: by redirecting the agent before it pursues unsupported actions, ECLoop lowers average token consumption by up to 12.1%.
