Vibeleaderboard
← All Intel
Intel / article

What Do Economic Value Benchmarks Tell Us

Source
epoch.ai
Author
Epoch AI publications
Date

Context

A new group of benchmarks claims to measure whether AI can do economically valuable digital work beyond coding. In a February 2026 report for Epoch AI, Florian Brand and Greg Burnham compared three of them. The Remote Labor Index (RLI), from CAIS and Scale AI, uses real freelance projects from Upwork that mostly need visual or multimedia output, such as 3D models, music arrangements and interior renderings, and freelancers reported spending 29 hours on them on average. APEX-Agents, from Mercor, gives about two-hour tasks in investment banking, management consulting and corporate law, set inside deliberately cluttered file systems. GDPval, from OpenAI, covers tasks from 44 occupations, mostly research and writing, that take professionals about seven hours.

Epoch argues that all three share a limit that their names hide. Each is a one-shot interaction: the AI gets a single static prompt, works and hands back one set of outputs, with none of the back-and-forth over goals and feedback that real work involves. Tasks are also cut off from any company's policies, processes or strategy, and the data is cleaned of proprietary and personal information. APEX-Agents adds some realism through its messy environments, while GDPval and RLI hand over exactly the information needed. The grading differs as well. GDPval and RLI ask human experts to compare the AI's output against a human reference, while APEX-Agents has an check outputs against a rubric of facts.

The scores Epoch reported as of February 2026 were spread far apart. GDPval was closest to saturation, with GPT-5.2 Pro at a 74% win-plus-tie rate against human experts. APEX-Agents topped out at 30%, from Opus 4.6. RLI's top score was 4%, which the authors attribute partly to its long tasks and to models' general weakness at work. Epoch also suspects all three under-measure models because of their scaffolds. RLI's Linux desktop cannot run Photoshop or Fusion360, so models fall back to GIMP or FreeCAD. APEX-Agents makes models juggle 79 tools, many of them wrappers around familiar Python libraries the models could likely use directly. These numbers are a snapshot: Mercor has since published APEX-Agents 1.1, so later APEX-Agents scores may come from a different version of the test.

Epoch's conclusion is that the benchmarks are useful but cover a narrow slice of the work, much as covers only part of software engineering. If models saturate them, the authors expect execution to speed up while planning, delegation and verification take up more of a worker's time. They describe the AI as more of an assistant for isolated tasks than a drop-in replacement for a whole job. Measuring that further step would take benchmarks built around complete tasks in the messy of real work.

Terms in this piece · Glossary
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
  • SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
More from Epoch AI publications
Recommended reads
Comments

Checking sign-in…

Loading comments…