AlignEval: Building an App to Make Evals Easy, Fun, and Automated
- Source
- eugeneyan.com
- Author
- Eugene Yan
- Date
Context
AlignEval makes the evaluator itself something to test. In Eugene Yan's description of the application, users inspect model inputs and outputs, label examples as passing or failing, and define one criterion. The application then compares an 's decisions with those labels.
Its optimization process adjusts the judge on one portion of the labeled examples and checks the result on a separate, previously unseen portion. That separation tests whether improvements carry beyond the examples used for adjustment. It parallels the point in Yan's evaluator review: an automated judgment needs its own assessment against human labels before it is trusted to assess another system.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
“The key insight is that aligning AI to human preferences is only half the battle. To build effective evals, we must also calibrate human criteria to AI output”
Eugene Yan
“This leads to two types of bad criteria. First, irrelevant criteria that are a waste of time, such as generic metrics (e.g., helpfulness) or very low probability defects (e.g., grammar, spelling). Second, unrealistic, unattainable criteria that the technology isn’t ready for, such as autonomous agents back in 2023.”
Eugene Yan
“What’s important is that we look at the data with an open mind, and not write criteria based on our priors. We should also resist the urge to prematurely define criteria. We need to first immerse ourselves in the data.”
Eugene Yan
“If our sample size is low (e.g., 20), the metrics can fluctuate a lot, even when we rerun the same LLM-evaluator. This is due to LLM stochasticity. We can improve stability by labeling more data. Aim for at least 50 labeled samples which also unlocks optimization mode.”
Eugene Yan
“With the small sample size (25 per split, 50 in total), this likely stems from the dev split being unrepresentative of the full data distribution, including the test split. Thus, the LLM-evaluator may overfit on the dev set and fail to generalize to the test set.”
Eugene Yan
Checking sign-in…
Loading comments…