Intel / article

HarnessTax: How Much Does the Harness Matter for Coding Agents?

Source: harnesstax.github.io
Author: matt_d
Date: Published Sep 16, 2026

Context

A coding 's manages its tools, and task execution. The HarnessTax study tested whether changing that software alters results for the same model. Across 30 sampled tasks from each of two benchmarks, with three attempts per task, the authors report similar success rates at substantially different costs.

For example, Anthropic's Fable 5 solved 97.8% of Lite attempts in Claude Code and 96.7% in Pi, while estimated cost per attempt was $1.33 versus $0.67. Costs use a fixed API price list, not subscription spending. This gives builders a reason to compare the whole model-and-harness setup. The results concern sampled benchmark tasks the models may have encountered in training, and do not establish which setup works best in interactive development.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.

Read the source harnesstax.github.io