Intel / article

The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior

Source: arxiv.org
Author: Xiangzhe Xu, Hamidreza Saghir, Qianhui Wu, Marc-Alexandre C\^ot\'e, Tong Wang, Kiran Lakkaraju, Kexin Pei, Xiangyu Zhang
Date: Published Aug 13, 2026

Context

How tools are presented to a coding can change how much work it takes to solve a task. This study compares six interfaces across three agents and 11,700 execution histories, keeping the available information and actions similar while changing their organization.

Compared with a bash-only baseline, an interface built around Python code achieved similar task performance with 41.6% fewer steps and 56.3% lower usage, a measure of model text consumption. The result makes interface design a concrete efficiency variable: adding capabilities is not the only way to improve an agent. These measurements concern the tested repository-fixing tasks and interfaces, rather than a universal reduction in runtime or bills.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Key quotes

“more structured low-level interfaces improve consistency across repeated attempts by up to 4.7 $\times$; natural-language search broadens repository exploration and increases access to relevant files by more than 11%”
Xiangzhe Xu et al.

“Python CodeAct-style interfaces achieve similar task performance with 41.6% fewer steps and 56.3% lower token usage.”
Xiangzhe Xu et al.

“By contrast, lightweight text-based cognitive-scaffolding tools, such as tools that let the agent record intermediate reasoning, have limited effect on actor behavior.”
Xiangzhe Xu et al.

“Prior work on agent tooling has primarily focused on expanding what agents can do, but has paid less systematic attention to how those capabilities are organized and exposed to the model. We refer to this latter design dimension as tool architecture.”
Xiangzhe Xu et al.

Read the source arxiv.org