Leaderboard Hacking: Preference-Based Model Evaluations are Vulnerable to Manipulation

A new paper from researchers at UC Berkeley, Università Bocconi, Stanford and MIT argues that arena-style AI leaderboards can be pushed around without any model getting better. On these sites, LMArena being the best known, a visitor types a prompt, reads answers from two anonymous models and picks the better one. The site turns those votes into a ranking with the Bradley-Terry model, a statistical method that scores each model by how often it wins, adjusted for the strength of the opponents it beat. Because a model's score depends on its rivals' scores, a vote in a match it never played can still raise it. The authors connect this to the Borda count, an election rule that Bradley-Terry implicitly reproduces and that voting theorists have long known can be gamed.

The authors tested three tactics in simulation, using a public LMArena dataset of about 100,000 votes across 55 models. In strategic voting, an attacker votes on a stream of new matchups and picks whichever winner helps the target most, which in matches without the target usually means voting for the weaker model. With at most 10,000 such votes, about a tenth of the dataset, models rose by up to 20 places, and for many models a gain of two or three places took under 2,000. Strategic nomination adds up to ten copies of an existing model that the target tends to beat more often than its rivals do. It lifted one model from 19th to 4th, though most gains were 1 to 7 places. Strategic prompting keeps every vote honest but concentrates the questions in an area where the target does well. Using prompt language as that area, Reka's reka-core climbed from 33rd to 18th on Hungarian prompts, and as few as 500 added prompts moved rankings. Nomination followed by prompting moved models by up to 45 places. Strategic voting only works if the attacker can tell which anonymous model is answering. The authors' identification tests picked out open-weight models with over 80% precision and three proprietary GPT models with roughly 40 to 50%, against a 25% base rate.

The paper shows that the ranking method can be gamed, not that anyone has gamed a live leaderboard. For ethical reasons the authors did not try to move models on the real rankings, and their dataset is far smaller than LMArena's main text leaderboard, which has more than 5 million votes. Their scaling tests suggest the number of manipulations needed grows roughly in proportion to the number of votes, so a bigger arena raises the price of an attack without ruling it out. The proposed defenses include verified voter accounts, flagging voters whose votes keep helping one model, stopping users from skipping matchups, and weighting clusters of similar prompts equally. Only one was measured: reweighting the Bradley-Terry objective along the lines of a harmonic Borda count cut the largest nomination gain from 15 places to 8, while the average possible gain rose slightly, from 2.74 to 2.81 places. This differs from the earlier Leaderboard Illusion study, which faulted how LMArena itself ran its arena, including undisclosed private testing by some providers. The new paper locates the weakness in the scoring method, so it would apply to any Bradley-Terry arena that lets people choose prompts, cast votes or submit models. A single arena position, particularly for a model sitting just below a cluster of close scores, is therefore weaker evidence of quality than it appears.