Intel / article

When Code Becomes an LLM Call: An Empirical Study of Call-Site Contracts in Agentic Systems

Source: arxiv.org
Author: Shihao Weng, Longtian Wang, Yang Feng, Mingfei Cheng, Jia Liu, Chao Shen, He Ye, Federica Sarro, Xiaofei Xie
Date: Published Oct 20, 2026

Context

When an framework hands part of its logic to a , such as choosing the next tool, judging whether an action is allowed, or writing code to run, the rest of the program still depends on that output being valid, but the model's behavior is not fixed by code, varies between runs and can be steered by its input. This empirical study argues the guarantee therefore has to come from the code around each call: conditions on when and with what input the model is called, constraints stated in the prompt, and checks on what comes back. The authors examined 496 such call sites in 40 popular open-source agentic systems, totalling 13.4 million lines of code in four languages, using a pipeline validated against 17,428 human judgments to trace where each output flows and to derive, without looking at the code, what each call site should guarantee.

The central finding is that having checks says little about whether they are adequate. Of model outputs, 48% reach control flow or execution, and 73% of those never reach a person. Although 82% of call sites are checked both before and after the call, 63% of the expected constraints go unmet and only 5.3% of call sites satisfy their whole contract. In prompts, 67% of constraints are only requested from the model rather than enforced. Ten recurring patterns explain 99% of the gaps; examples include an empty section passing a format check and being written into a STORM article, and Gemini CLI trusting a no-change flag from the model. The authors showed 38 of the gap mechanisms trigger at runtime, and a small code fix enforcing the expected constraint blocked the original attack in all 38, though a reworded variant still got through in three.

The expected contracts are the authors' reconstruction, which other analysts might draw slightly differently, the runtime triggers were tested with one substitute model and at most ten framings of each payload, and the authors note that closed-source agents and frameworks may differ.

Key takeaways · AI-distilled

The study frames each LLM call in an agent as needing a contract of pre-conditions, prompt constraints and post-conditions, since model output varies across runs and can be steered by input. It examined 496 call sites across 40 open-source agentic systems.
The authors report that 48% of LLM outputs flow into control flow or execution, and 73% of those never pass a human before taking effect.
In the studied systems, code enforces 78% of constraints, but 67% of prompt-level constraints are only requested of the model, not enforced.
The authors find checks common but weak: 82% of call sites validate before and after the call, yet 63% of expected constraints go unmet and only 5.3% of call sites meet their full contract. Ten recurring patterns explain 99% of the gaps.
The authors found 38 gap mechanisms that trigger at runtime and report that a small code fix enforcing the expected constraint stopped all 38.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.

Read the source arxiv.org