alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
Why it matters
If you're building LLM-as-judgeUsing one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.Full definition → evaluators, this breaks down alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → techniques, fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → approaches, and the concrete failure modes of using LLMs to grade LLMs — helping you decide when to trust automated evaluation and how to make it more reliable.
Key quotes
“Most folks have human annotators as the baseline. Here, we aim for the LLM-human correlation to match human-human correlation. Compared to human annotators, LLM-evaluators can be orders of magnitude faster and cheaper, as well as more reliable.”
Eugene Yan
“Pairwise comparison chooses the better of two responses or declares a tie. It’s typically used—and more reliable—for subjective evals such as persuasiveness, tone, coherence, etc. Studies show that pairwise comparisons lead to more stable results and smaller differences between LLM judgments and human annotations relative to direct scoring.”
Eugene Yan
“I tend to be skeptical of correlation metrics. They don’t account for chance agreement and thus could be overoptimistic (though Cohen’s $\kappa$ is an exception).”
Eugene Yan
“Across both tasks, the results showed that as the LLM-evaluator increased in parameter count, it becomes more accurate at identifying harmful behavior as well as classifying it.”
Eugene Yan
“The findings showed that applying Chain-of-Thought (CoT) improves the accuracy of LLM-evaluators. Furthermore, the trends suggest that LLM-evaluators larger than 52B can be competitive with preference models finetuned on human feedback.”
