The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior
- Source
- Xiangzhe Xu, Hamidreza Saghir, Qianhui Wu, Marc-Alexandre C\^ot\'e, Tong Wang, Kiran Lakkaraju, Kexin Pei, Xiangyu Zhang
- Author
- Xiangzhe Xu, Hamidreza Saghir, Qianhui Wu, Marc-Alexandre C\^ot\'e, Tong Wang, Kiran Lakkaraju, Kexin Pei, Xiangyu Zhang
- Date
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Separates what an can do from how its tools are shaped, and shows the shape itself changes behavior, which is the part most agent harnesses tune blind.
“more structured low-level interfaces improve consistency across repeated attempts by up to 4.7 $\times$; natural-language search broadens repository exploration and increases access to relevant files by more than 11%”
Xiangzhe Xu et al.
“Python CodeAct-style interfaces achieve similar task performance with 41.6% fewer steps and 56.3% lower token usage.”
Xiangzhe Xu et al.
“By contrast, lightweight text-based cognitive-scaffolding tools, such as tools that let the agent record intermediate reasoning, have limited effect on actor behavior.”
Xiangzhe Xu et al.
“Prior work on agent tooling has primarily focused on expanding what agents can do, but has paid less systematic attention to how those capabilities are organized and exposed to the model. We refer to this latter design dimension as tool architecture.”
Xiangzhe Xu et al.
articleArchitecture as Capability Equalizer for Coding AgentsArquimedes Canedo
articleIs Bash All You Need? An Empirical Study of Tool Interfaces for Enterprise Digital Worker AgentsHazel Mak, Susheel Suresh, Sahil Bhatnagar, Barry Wang, Chhaya Methani, Alejandro Gutierrez Munoz
articleHarness Engineering in LLM Tool Use via Agent-Native Reusable Tool PrimitivesHaibo Jin, Suijin Wang, Xucheng Yu, Haojing Luo, Haohan Wang
Checking sign-in…
Loading comments…