Intel / article

What Stops a Small Language Model From Driving a Database Agent

Source: arxiv.org
Author: Cevheri Bozoglan, Yusuf Gundogdu, Abdullah Kaya, Koray Sirin
Date: Published Sep 21, 2026

Context

A database can fail because its tools impose contradictory requirements. In a study of 8,199 runs, researchers tested 39 locally served models and one hosted control in a SQL client's agent mode. They report that 75.7% of failures attributed to models occurred after the run had already invoked a tool.

The authors report that inspecting the actual arguments sent to tools exposed five server defects. One required a field in one tool, prohibited it in a related tool, and subsequently failed the run when it was absent. The authors report improvements after server changes without changing models or prompts. This makes the surrounding software part of the explanation for this system's failures; attributing every unsuccessful task to weak model reasoning would miss repairable causes.

Key takeaways · AI-distilled

The study drove 39 models plus one hosted control against a production SQL-client agent over 11 days: 8,199 runs, 110,711 ledger events, 14,008 refused tool calls.
Of 2,100 model-attributed losses, 75.7% came from runs that had invoked at least one tool, and that majority held in 99.7% of clustered resamples across models, not just runs.
Within failures, transport (tool used but no deliverable came through) was the largest class at 36.2%; capability (no tool invoked at all) was smallest at 17.3% - though the authors flag this ordering as corpus-specific, holding in only 74.5% of resamples when clustered by model.
Production ledgers logged refusal codes but never the model's actual arguments, hiding five server defects for ten days - one required a field on one tool while forbidding it on the sibling tool that composed it. Fixing all five moved six models by 6-21 cells out of 30.
A benchmarking confound: with no cap, one 7.1GB model was admitted at its full 262,144- window and consumed 51GB on a 64GB machine, producing runs indistinguishable from a timeout in ordinary logs.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Read the source arxiv.org