Intel / article

LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent

Source: arxiv.org
Author: Han Li, Zhemin Fang, Rili Feng, Yingqi Zhao, Jiaheng Liu, Pengfei Gao, He Ye, Dayi Lin, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
Date: Published Aug 4, 2026

Context

LoopsBench tests whether a coding can extend software while keeping earlier work functioning. Its 112 tasks contain separately testable development units linked by prerequisites. Tests become available as their prerequisites are satisfied, and completed units remain subject to checks so later edits can reveal regressions. This makes the benchmark useful for examining how an agent sequences and preserves work across a long assignment. The strongest tested configuration, Anthropic’s Opus 4.7 with Claude Code and an outer continuation loop, completed 25% of tasks. That result applies to the evaluated configuration and conditions.

Key takeaways · AI-distilled

LoopsBench reframes coding-agent from engineering to LOOP engineering: 112 long-horizon tasks across 8 languages, each a dependency DAG of separately testable development units.
The flow-aware runtime releases tests along the ready frontier and keeps completed nodes as standing regression obligations - so an agent can lose ground it already won.
Even the strongest configuration (Opus-4.7 + Claude Code with outer continuation) resolves only 25% of tasks - sustained multi-unit development remains far from solved.
Agents recorded plans recover only part of the true prerequisite DAG, and regression events stay visible across every loop profile evaluated.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.

Key quotes

“Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development.”

“LOOPSBENCH comprises 112 tasks from authentic sources spanning 8 programming languages and 9 domains.”

“The strongest configuration, Opus-4.7 with Claude Code and outer continuation, resolves 25.00% of tasks.”

Read the source arxiv.org