Introducing The Ai Productivity Index For Accounting

Running each task eight times shows that even the best model solved only 2.6% of tasks every time, and budget sensitivity varied sharply by model. Reliability, not single-run accuracy, is the gap for agents doing multi-step work.

• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.

• articleMonty Engineering Deep Dive
• articleHuman Baselines For Benchmarks Ai Now Outperforms Junior Accountants
• articleHow We 20x Llm Eval Throughput
• articleIntroducing Apex Agents 1 1

• articleHuman Baselines For Benchmarks Ai Now Outperforms Junior AccountantsMercor research blog
• articleAPEX-AccountingJulien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen
• articleIntroducing Apex Agents 1 1Mercor research blog

Checking sign-in…

Loading comments…