Ai2 uses psychometrics to audit what LLM safety benchmarks measure
- Source
- x.com
- Date
Do LLM safety & capability evals measure what they claim to? We built BenchMIRT to audit them + see which model abilities their Qs actually test. On BBQ, a social-bias eval, it found the Qs distinguished models more by reasoning ability than safety. 🧵 https://t.co/BQ8WPTu5nR
BenchMIRT builds on Item Response Theory (IRT), a technique from psychometrics for measuring abilities from patterns of test responses. The idea: not every question tells you the same amount. Some are harder; some better distinguish stronger models from weaker ones.
We trained BenchMIRT on results from 100 LLMs across 16 benchmarks & 34K+ questions. We didn’t tell it which evals measured what. Two dominant dimensions consistently emerged: general reasoning + safety.
BenchMIRT works at multiple levels: ◙ For models, it estimates strength on the capabilities reflected in the benchmark set. ◙ For questions, it estimates difficulty & how strongly each Q distinguishes models along those capabilities.
Context
A low score on a social-bias test may partly reflect difficulty with reasoning rather than bias. Ai2's BenchMIRT studies patterns of correct and incorrect answers to estimate which abilities each question actually tests. Trained on results from 100 models across 16 benchmarks, it found that BBQ, a social-bias usually grouped with safety tests, tracked general reasoning far more closely than safety. Research explanation.
This gives evaluators a way to investigate what drives an aggregate score before treating it as a clean measure of safety. Ai2 states limits: the models were all released by March 2025, and the abilities found depend on the chosen benchmarks, so a different collection could surface different ones. Scope and limitations.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleBenchMIRT: What are LLM benchmarks actually measuring?allenai.org
articleEvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language ModelsXinning Li, Kemunto Ochwang'i, Aryasomayajula Ram Bharadwaj, Alexandra Souly, Robert Kirk
articleFinding Flawed Questions in AI Benchmarks with PsychometricsStanford AI Lab
Checking sign-in…
Loading comments…