Building Reliable Data Analytics Agents: Lessons from the KDD Cup
- Source
- developer.nvidia.com
- Author
- Jiwei Liu
- Date

Context
A team from NVIDIA placed second in the KDD Cup 2026 Data Agents competition, and by its own account the gain came from constraining the agent, not from a more capable model. In their write-up, the task was answering questions over mixed sources (databases, CSV and JSON files, documents, videos) with a small fixed , so the surrounding software, called the harness, was what they could improve.
Their main choices were to load all structured data into one SQLite database queried through two functions, schema() and (), and to run a preflight step briefing the agent on tables, join keys and data quirks before it starts. They also limited the toolset and routed document reading through a helper so raw text stayed out of the agent's . The authors say this cut routing errors and wasted turns.
They logged every run so an inspector agent could sort failures, and warn that tuning against a benchmark can overfit. The authors say the playbook is shaped by this competition, not a general recipe.
- NVIDIA's KGMON team placed second in the KDD Cup 2026 Data Agents competition, where every team had to use the same small, fixed LLM. That made the harness around the model, not the model itself, the main thing to optimize.
- KGMON loaded the task's CSV and JSON files into its existing SQLite database so the agent had one query surface, exposed through two custom functions, schema() and sql(query). The team says this cut routing failures and wasted turns.
- Before the main loop, a read-only schema-scouting step briefed the agent on tables, likely join keys, look-alike fields, units, null patterns and row-grain issues, which the authors credit with fewer wrong-column and missed-join errors.
- The agent could not read whole documents. It searched with capped previews or regex, then sent relevant chunks to prose_helper, a separate -0 LLM call that returned a short answer or extracted a SQL table, keeping raw prose out of the main context.
- The team warns that autonomous improvement loops overfit through hardcoded training examples, contradictory prompt instructions and brittle postprocessing. Its guard: held-out tasks, prompt audits, trace review and human approval before a change is kept.
- Large language model — A model trained on large amounts of data to process and generate language.
- Database and SQL — Persistent structured storage queried and changed through explicit operations.
- Context and context window — Context is the information available for a generation; the context window is the capacity available to hold it.
- Sampling controls — Parameters that shape how deterministic or varied model generation becomes.
Checking sign-in…
Loading comments…


