1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life
Source
youtube.com
Author
AI Engineer
Date
Why it matters
Tells you where to spend evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → effort: deployment controls like human review and monitoring track harm more than model strength, and the evaluator itself needs checking.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
videoWhy AI Writing Still Reads Like Slop — Louis-François Bouchard, Towards AI
videoHow to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
videoThe AI Content System Behind 200M Views — Sarah Chieng, Cerebras & Cova Lee, OpenAI
videoDoes Code Quality Still Matter? It Depends on This — Max Kanat-Alexander, Capital One
