Expert judgment in the reward function beats scaffolds on text-to-SQL

A database query can return the expected answer for the wrong reason. In a guest post by researchers at UIUC and Bridgewater, published by Thinking Machines, a generated query averaged the prices of individual line items instead of whole-order totals, yet matched the reference result because each order happened to contain a single line item. Rewarding such answers teaches a model unreliable SQL, the language used to query databases. The researchers corrected training examples and added a solver-based check for whether two queries are truly equivalent, plus a check that the query follows the background information supplied with the question. Their report explains the reward changes.

The resulting model scored 91.37% on the cleaned Arcwise-Plat-SQL evaluation with one generated answer. Voting across 16 candidates raised that to 92.97%. The useful finding is that expertise can improve what training rewards, rather than only adding more steps around an unchanged model. Results.