Intel

Privacy Terms Submit Map Use with your agent

Gaming LM Arena with watermarked crowdsourcing — Intel — VibeLeaderboard

← All Intel

Clip / Education

Gaming LM Arena with watermarked crowdsourcing

From Benchmaxxing: The Gap Between Benchmark Scores and Reality · ≈12:02

“Another thing you can do that I've heard stories of is you can actually hire a crowdsource army to vote for you in Elmarina since Elmarina basically does no filtering of their workforce.”

“You have your model include a watermark that tells the crowd who to vote for.”

What’s in it

Exposes a real trick to rig crowdsourced AI benchmark voting
Explains how model watermarks defeat anonymized benchmark platforms
Reveals why platforms like LMArena are vulnerable to manipulation

Watch this moment ↗

Clip transcript

benchmax, you just cannot care about that. Another thing you can do that I've heard stories of is you can actually hire a crowdsource army to vote for you in Elmarina since Elmarina basically does no filtering of their workforce. And you might say, well, we anonym, you know, Elmarina anonymizes. So how are they going to know who to vote for? That's actually quite simple. You have your model include a watermark that tells the crowd who to vote for.