Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Source
Qian Wang, Zhanzhi Lou, Zhenheng Tang, Nuo Chen, Bingsheng He
Author
Qian Wang, Zhanzhi Lou, Zhenheng Tang, Nuo Chen, Bingsheng He
Date
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
If you're building LLM-as-judgeUsing one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.Full definition → pipelines, this shows that picking the right auditor model per bias type—not just the most 'unbiased' model overall—meaningfully improves judgment accuracy (0.884 vs 0.824/0.805), giving you a concrete lever to reduce sycophancy, bandwagon, authority, and distraction biases in your evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → setup.
Key quotes
“standalone bias resistance does not predict audit effectiveness”
