Contamination Inflates Scores but Rarely Reorders Large Language Model Leaderboards
Source
Xingyao Xiao (Stanford University), Yihong Cheng (City University of Macau)
Author
Xingyao Xiao (Stanford University), Yihong Cheng (City University of Macau)
Date
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Why it matters
Contamination is often treated as invalidating leaderboards outright. This work shows it inflates scores but rarely reorders which model beats which, meaning relative leaderboard rankings stay more trustworthy than raw contaminated scores suggest.
articleMemorization Diagnostics for Code LLMs Should be Scale-AwarePrateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djir\'e, Xunzhu Tang, Yewei Song, Iyiola Emmanuel Olatunji, El Hacen Diallo, Jacques Klein, Tegawend\'e F. Bissyand\'e
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li