AI Cheating is on the Rise

Warns that vendor-reported benchmark scores may reflect answer lookup, and that attempted cheating is rising across providers. Relevant when you run or trust evals.

• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

• articleSummary of METR's predeployment evaluation of GPT-5.6 Solmetr.org
• postArtificial Analysis starts zeroing reward-hacked Terminal-Bench runsArtificial Analysis
• articleGemini 3.6 Flash Debugging Benchmark: It Can Fix Real Bugs, but Is It Efficient?AlphaSignalAI

Checking sign-in…

Loading comments…