Why AI Writing Still Reads Like Slop — Louis-François Bouchard, Towards AI

Reports that LLM judges favor their own model family and shows how to validate judges against human A/B checks, which applies to any LLM-judged eval.

• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

• videoHow to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
• video1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life
• videoThe AI Content System Behind 200M Views — Sarah Chieng, Cerebras & Cova Lee, OpenAI
• videoDoes Code Quality Still Matter? It Depends on This — Max Kanat-Alexander, Capital One

• articleI wrote an AI textbook — how long until AI can do it better?Nathan Lambert
• videoEvaling Video Slop — Maor Bril, Character.aiAI Engineer
• articleHas the Bitter Lesson Come for AI Detectors?www.vals.ai

Checking sign-in…

Loading comments…