Intel / article

Fidelity Collapses with Table Size, Not Turn Count: Delivery Architecture in a Deployed Tool-Using Agent

Source: arxiv.org
Author: Tung Thanh Hoang
Date: Published Nov 3, 2026

Context

This paper asks where a tool-using 's data errors come from, and its answer, measured in a deployed award-travel agent, is the size of the table the model has to carry, not the length of the conversation (arXiv). The author compared two designs on identical frozen tool results. In the one called SYNTHESIS, the full set of records goes into the model's and the model answers follow-up questions from memory, re-emitting rows each turn. In BYPASS, the system actually deployed, records stay in session state, follow-ups are turned into arguments for a deterministic filter, and the matching rows go to the user over a typed side channel instead of through the model's reply.

Across 24 dialogues (339 scored turns per condition), the abstract reports 92.3% of BYPASS turns fully correct against 45.7% for SYNTHESIS. SYNTHESIS was near-perfect at 10 rows or fewer but fell to 8% or less beyond about 74 rows. In a controlled test that grew the table step by step, the breaking point rose with model capability, from under 10 rows for GPT-4o-mini to about 198 or more for Gemini 2.5 Pro, while BYPASS stayed flat for all four models tested. Beyond 100 rows, the author estimates each correct SYNTHESIS answer cost roughly 200 times BYPASS's flat $0.012.

The practical point is about agent architecture: keeping structured results out of the model's own output, which builders often do so a UI can render them, also protects their accuracy. The evidence is one production system and one domain, presented as a non-archival workshop paper at EMNLP 2026.

Key takeaways · AI-distilled

The study compares two architectures on identical frozen payloads in a production award-travel agent: SYNTHESIS puts full record sets in the planner's context, while BYPASS keeps records in session state and answers follow-ups through a deterministic filter.
Across 24 dialogues and 339 scored turns per condition, BYPASS got 92.3% of turns fully correct versus 45.7% for SYNTHESIS, a paired gap of 46.6 points.
The failure tracks table size, not conversation length: SYNTHESIS was near-perfect at 10 rows or fewer and fell to 8% or less beyond about 74 rows, while BYPASS stayed size-flat for all four models tested.
Beyond 100 rows, each correct SYNTHESIS answer cost roughly 200x BYPASS's flat $0.012, according to the paper, which argues typed side-channel delivery is a correctness decision, not only a rendering one.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Read the source arxiv.org