Intel / article

Auditing and Repairing LLM-as-Judge Failures in a Production Text-to-SQL Pipeline

Source: arxiv.org
Author: Haowei Liu, Hsin-Tai Wu, Yi Fang
Date: Published Sep 28, 2026

From the Daily Brief

Researchers audited the LLM judge deployed in a production text-to-SQL pipeline and found it barely agreed with human graders: a Cohen's kappa of 0.04, close to chance. The judge over-flagged 77% of correct results, mostly through what the authors call grade hallucination.

The repair is as notable as the failure. A self-hosted Qwen3.6-27B judge matched Claude Opus 4.7 at roughly 1/300th of the cost, so fixing the judge did not require the most expensive model.

The lesson for anyone grading agent output automatically is to measure judge agreement against a human-labeled sample before trusting its scores. A judge that passes a quick look can still be producing noise that steers every downstream decision. A companion study this week found the MARCH multi-agent code judge called ties on 78 to 95% of comparisons, another reminder to validate judges directly.

Read the 2026-09-29 Brief →

Context

An AI checker can reject correct database queries by inventing requirements the user never supplied. In a production text-to-SQL audit, a judge using OpenAI's GPT-4o-mini expected a small canonical set of grade codes and flagged rarer values in query results as invalid, even though the question never asked for those rows to be excluded.

On a sample deliberately enriched for disagreements, it incorrectly flagged 77.1% of cases that human reviewers considered faithful. That percentage describes the selected sample, not all production queries. Removing the result preview reduced this particular failure, but the tested prompt changes also increased missed errors. The study shows why a checker needs against human judgments: a plausible explanation for rejecting a query can itself be the mistake.

Key takeaways · AI-distilled

Measured against two-author gold labels, a deployed gpt-4o-mini text-to-SQL judge reached Cohen's kappa of only 0.04 on a disagreement-enriched set and 0.42 on a random spot-check, over-flagging 77.1% of faithful cases.
A self-hosted Qwen3.6-27B judge reached kappa 0.72, close to Claude Opus 4.7 at 0.71, at roughly 1/300 of the cost per call; the authors note the head-to-head is underpowered at n=96.
Pairing a weak judge with a strong one lowered agreement, while three strong judges with unanimity routing reached kappa 0.79 and auto-handled 89.7% of cases.
Applied to BIRD-financial, the same audit recipe flagged 25.5% of the expert-written gold SQL queries as candidate gold-label issues under the authors' protocol.

Terms in this piece · Glossary

eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.

Read the source arxiv.org