Vibeleaderboard
← Glossary
Glossary · AI and agent workflows

Model judge

A model asked to read an output and say whether it meets a criterion.

A model judge reads for meaning and scales, and its verdict is generated text: imperfect evidence whose reliability has to be measured against a person's readings on the same cases, with disagreements resolved against the source evidence, since the person can be wrong too. It can prefer longer responses and, when comparing, whichever comes first or last. A judge may share errors with the model whose work it judges, most plainly within the same model or family, and a model reviewing its own output can find real problems without serving as independent evidence that there are none. A reviewer that checks actions for danger is not a judge of correctness.

See how the idea fits

Who can check the work, and what can each establish? →
Definition sources