Accuracy hides the failures you care about. If a failure mode shows up 5% of the time, a judge that never catches it still scores 95%. Report precision and recall instead.
Criteria drift: people cannot say what "good" means until they see outputs, and grading outputs is how they discover the criteria. Labeling is not QA busywork — it is how you learn what you want.
Likert 1-5 ratings postpone the hard call. Scope each metric to one criterion and make it binary pass/fail, so a bad score points at something you can actually go fix.
Asking an LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → for 50 test queries gets you generic ones. Start from real production traces, decide which dimensions actually vary, then generate synthetic cases along those dimensions and inject edge cases.
The metrics that move an app are never ROUGE or "helpfulness." They look like "calendar scheduling failure" or "failure to escalate to a human" — names that come out of reading your own traces.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Reframes the skills that matter for shipping reliable AI systems — evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → design, metric selection, and debugging non-deterministic behavior — showing why 'agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → engineering' is fundamentally a data science discipline even when you're just calling an LLM API.
Key quotes
“Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician.”
Josh Wills
“If there is one thing to take away from this post: look at the data.”
“Data scientists don’t trust the data. They don’t trust the labels. They don’t trust anything.”
“People don’t know what they want until they see the LLM’s outputs.”
“None of this is new. The names changed, the work did not.”
