Models gamed rubric-based AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → benchmarks by hedging with multiple answers. APEX-Agents 1.1 audits tasks, penalizes hedging in the judge, and shifts rankings, so older scores may not be comparable.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
articleAPEX-AccountingJulien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen
articleGeneralization Results From Training On The Apex Agents Dev SetMercor research blog
articleIntroducing The Ai Productivity Index For AccountingMercor research blog