Intel / article

Beyond the Model: Demystifying Harness Effects in Software Engineering Agents

Source: arxiv.org
Author: Haichuan Hu, Quanjun Zhang, Shengcheng Yu, Zhifei Chen, Tianyu Luo, Chunrong Fang, Zhenyu Chen, Liang Xiao
Date: Published Sep 29, 2026

Context

A coding agent's surrounding software can help or hinder it depending on the work. The NanoHarness study examines the tools, context handling and delegation around a model. In its repository-generation tests, structured tools and subagents assigned specific jobs produced steadier improvements than general-purpose delegation; compressing the working history cut prompt tokens sharply but lowered scores. The broader comparison also found that extra infrastructure offered smaller gains for stronger models on issue repair, while stronger models could benefit on more open-ended repository tasks. The evidence favors matching the agent's tools and coordination to its assignment, rather than treating additional machinery as an automatic upgrade.

Key takeaways · AI-distilled

The study tested mini-SWE-agent and OpenCode with ten Qwen and DeepSeek open-weight models on Pro, ProgramBench and GitTaskBench, and found value depends jointly on model capability and task type.
Complex harnesses gave diminishing gains on SWE-style issue repair as models got stronger, but could still help stronger models on more complex, open-ended repository-level tasks.
In component ablations on ProgramBench, structured and task-specific subagents gave the most stable gains, while and general-purpose subagents could hurt repository-generation performance.
Combining the helpful components in their NanoHarness lifted results over mini-SWE-agent by 7.37 points on Qwen3.7-Max and 6.21 points on DeepSeek-V4-Pro, recovering most of the gains of product-level harnesses, per the authors.

Terms in this piece · Glossary

SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
context compaction — Summarizing an agent's earlier conversation to free room in the context window so a long session can keep going.

Read the source arxiv.org