• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

LLM judges score holistically and drift between runs. Deriving explicit criteria from the task guidelines and applying them one at a time makes judgments more stable and lets you see which criterion drove a verdict.

• articleCommit0 Library Generation From Scratch 2024 12 02
• articleSeacrowd A Multilingual Multimodal Data Hub And Benchmark Suite For Southeast Asian Languages 2024 06 14
• articleFishing For Magikarp Automatically Detecting Under Trained Tokens In Large Language Models 2024 05 08
• articleBpe Stays On Script Structured Encoding For Robust Multilingual Pretokenization 2025 05 30

• articleUnderstanding the 4 Main Approaches to LLM Evaluation (From Scratch)Sebastian Raschka, PhD
• articleEvaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)Eugene Yan
• clipWhy judge-model quality review has a ceilingAI Engineer

Checking sign-in…

Loading comments…