Intel / article

Building Reliable Data Analytics Agents: Lessons from the KDD Cup

Source: developer.nvidia.com
Author: Jiwei Liu
Date: Published Oct 8, 2026

Context

A team from NVIDIA placed second in the KDD Cup 2026 Data Agents competition, and by its own account the gain came from constraining the , not from a more capable model. In their write-up, the task was answering questions over mixed sources (databases, CSV and JSON files, documents, videos) with a small fixed , so the surrounding software, called the , was what they could improve.

Their main choices were to load all structured data into one SQLite database queried through two functions, schema() and sql(), and to run a preflight step briefing the agent on tables, join keys and data quirks before it starts. They also limited the toolset and routed document reading through a helper so raw text stayed out of the agent's . The authors say this cut routing errors and wasted turns.

They logged every run so an inspector agent could sort failures, and warn that tuning against a benchmark can overfit. The authors say the playbook is shaped by this competition, not a general recipe.

Key takeaways · AI-distilled

NVIDIA's KGMON team placed second in the KDD Cup 2026 Data Agents competition, where every team had to use the same small, fixed LLM. That made the harness around the model, not the model itself, the main thing to optimize.
KGMON loaded the task's CSV and JSON files into its existing SQLite database so the agent had one query surface, exposed through two custom functions, schema() and sql(query). The team says this cut routing failures and wasted turns.
Before the main loop, a read-only schema-scouting step briefed the agent on tables, likely join keys, look-alike fields, units, null patterns and row-grain issues, which the authors credit with fewer wrong-column and missed-join errors.
The agent could not read whole documents. It searched with capped previews or regex, then sent relevant chunks to prose_helper, a separate temperature-0 LLM call that returned a short answer or extracted a SQL table, keeping raw prose out of the main context.
The team warns that autonomous improvement loops overfit through hardcoded training examples, contradictory prompt instructions and brittle postprocessing. Its guard: held-out tasks, prompt audits, trace review and human approval before a change is kept.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Read the source developer.nvidia.com