Vibeleaderboard
← Glossary
Glossary · AI and agent workflows

Model grader

Also: LLM as judge, model grader, model-graded eval, AI judge

A model used to judge whether another model's output meets a criterion, in place of or alongside a person.

A model grader can reduce the cost of evaluating many outputs, and its judgments are themselves generated text. One provider's guidance says to check a grader's verdicts against human labels and to watch for length and position bias. A score from a grader that has not been checked is a measurement of the grader as much as of the graded.

See how the idea fits

What does a benchmark score tell you? →
Definition sources