A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Change a prompt, swap a model, tweak a harness — did it get better? With nondeterministic output you can't tell from a few tries. An eval fixes a task set and a scoring method (exact match, passing tests, or an LLM judging outputs) so quality becomes a number you can track and regress against.
"Write evals before shipping agents" is now standard advice for the same reason tests-before-refactor is. Public evals are benchmarks; private ones encode what YOUR product means by good — and the private ones are usually worth more.