Human Baselines For Benchmarks Ai Now Outperforms Junior Accountants

A human baseline shows frontier models beating junior CPAs on structured month-end close tasks, at roughly a tenth of the cost. The authors caution that the tasks favor what models do best.

• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.

• articleMonty Engineering Deep Dive
• articleHow We 20x Llm Eval Throughput
• articleIntroducing The Ai Productivity Index For Accounting
• articleIntroducing Apex Agents 1 1

• articleIntroducing The Ai Productivity Index For AccountingMercor research blog
• articleAPEX-AccountingJulien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen
• articleCrosscheck Benchmarking Ai Models In The Real WorldLinkedIn Engineering

Checking sign-in…

Loading comments…