GDP.xlsx: Benchmarking Frontier Models on Professional Workbooks
- Source
- arxiv.org
- Author
- Suhaas Garre, Sushant Mehta, Edwin Chen
- Date
GDP_xlsx is a new benchmark of 70 workplace requests written by professionals in 13 domains, each paired with the actual workbook or CSV it refers to and an expert rubric of binary criteria. Together the files hold 2.18 million populated cells and the rubrics 791 criteria. Agents work in a container with spreadsheet tools, and an agentic judge that can open the resulting file applies the rubric, with completeness and exclusivity criteria requiring set-valued answers to be complete and contain nothing extra. Models are ranked by strict pass@1, the share of attempts that meet every criterion. Across 20 configurations, including frontier models from Google, Anthropic and OpenAI, Gemini 4 Argon scored highest at 38.3% and the median was 19.3%. The authors release the evaluation harness and three public tasks and hold back the other 67 to keep scores free of contamination, so outside teams can run their own agents through the harness without seeing the scored tasks.

Context
On a new of questions that professionals ask about their own spreadsheets, no tested answered reliably: the best configuration, Google's Gemini 4 Argon, fully satisfied the expert's checklist on 38.3% of attempts. For GDP.xlsx, researchers at Surge AI collected 70 requests written by professionals in 13 fields, from manufacturing and construction to healthcare and finance, each built around a workflow in which AI models had failed the author. Each request comes with the complete file it refers to, 58 Excel workbooks and 12 CSV files holding 2.18 million populated cells in all, and with a rubric of yes-or-no criteria, 791 across the set. The requests are short, written as the author would ask a colleague, and 58 of the 70 never mention a tab or sheet, so the agent first has to find where the answer lives.
The difficulty comes from where workbooks keep their evidence and from how strictly answers are graded. In 21 tasks the answer depends on layout or formatting: a fill color marking a record as invalid, a comment changing how a value should be read, a filter hiding a row, a diagram pasted onto a sheet. Each agent works in a container with Python spreadsheet libraries, LibreOffice and text recognition for images, with up to 60 minutes and 200 turns and no network access beyond its model provider. A second agent acting as judge reads the final answer, can open the file and run code to verify it, and marks each criterion. In 39 tasks the rubric demands both completeness, every item of a set, and exclusivity, nothing outside it. An attempt passes only if it meets every criterion, and the score, strict pass@1, is the share of passing attempts averaged over tasks, with five attempts per task.
Every one of the 20 configurations failed most attempts. After Gemini 4 Argon came Anthropic's Claude Opus 5.5 at 30.3% and Claude Sonnet 5.5 at 29.1%; nothing else passed 25%, and the median was 19.3%. The four failures the authors examine all left out evidence that was in the file. Muse Spark 1.3 listed six of the nine tenancies a rent request asked for, with no wrong entries, but missed the three that qualified only through break clauses recorded on another tab. OpenAI's GPT 6 Astra gave three of the eight patient codes a clinical data dictionary held for housebound patients. Gemini 3.8 Flash named the wrong equipment cabinet because the right one was labeled only in a diagram pasted onto a sheet. The authors point out that a few lines of Python can list every tab, comment, filter and embedded image in a workbook, so the agents had the means to find what they missed; the misses look like failures to search exhaustively and to decide when a set is complete.
The scores carry limits worth knowing. With only 70 tasks, sampling error is large: the authors bound it at 5.8 points for the top score, and the five configurations bunched between 22.0% and 23.1% differ by far less than that. Only three tasks are public; the other 67 are held back to keep them out of training data, which protects the leaderboard but leaves outsiders unable to inspect most of the material. The judge is itself a model, and the paper neither names it nor reports how often its verdicts match a human grader's. Strict grading means one missed item fails an otherwise correct answer, so the scores measure fully dependable answers rather than partial progress. GDP.xlsx also grades only the written answer, not any file the agent produces; MBABench-2.0 covers the other side of spreadsheet work, building financial workbooks from scratch.
- GDP_xlsx has 70 workplace requests written by professionals in 13 domains, each paired with the workbook or CSV it refers to (2.18 million populated cells in total) and an expert rubric of binary criteria, 791 across the set.
- An attempt passes only if it meets every rubric criterion, and set-valued answers must be complete with nothing extra. An agentic judge that can open the output file applies the rubric.
- Across 20 model configurations, including frontier models from Google, Anthropic and OpenAI, the top score was Gemini 4 Argon at 38.3% strict pass@1 and the median was 19.3%, per the abstract.
- The authors release the and three public tasks, holding the other 67 back to protect the leaderboard from contamination.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleMBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple VerifiabilityJulian Poeltl, Thomson Yen, Patrick Shea, Shubham Anil Varma, Dhruv Rastogi, Zihao Chen, Mengshi Zhang, Hongseok Namkoong
postGDPx: a frontier agentic benchmark where the top score is 12%SemiAnalysis
articleSWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding BenchmarkSoham Dan, Jeff Da, Sanyam Bhutani, Miguel Romero Calvo, Ivan Guevara, Fernando Carabedo, Jose Bernal, Julio Reyna, Luis Leandro Monsalve Mendez, Ying Liu, Daniel Yue Zhang
Checking sign-in…
Loading comments…