EverydayBench: Evaluating Satisfactory Completion of Live User-Facing Digital Work
Source
arxiv.org
Author
Fangrui Huang, Bingyang Wang, Chuhan Li, Yijiang Li, Arpandeep Khatua, Austin Shen, Riki R Z Choi, Cuiyuanxiu Chen, Jin Huang, Zachary Robertson, Kate Lin, Akshay Paruchuri, Sijun Tan, Xin Liu, Zhengmin Ni, Sanmi Koyejo, Ehsan Adeli
Date
Why it matters
Live-web agent tasks have shifting valid answers; this benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → fixes success criteria up front and checks current evidence, giving a more trustworthy way to evaluate browsing agents.
Key takeaways · AI-distilled
EverydayBench has 77 human-authored tasks across eight domains of everyday life and work. Each task fixes its success criteria before execution without prescribing an answer or workflow, so agents choose their own outcomes.
Scoring combines three checks: deterministic tests of explicit constraints, verification of claims against the pages the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → cited, retrieved again at evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → time, and a rubric for the quality of the final deliverable.
Across 23 agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tied for the highest strict pass rate at 54.55%. In a controlled effort sweep, GPT-6 Sol went from 29.87% at low effort to 45.45% at max effort.
No model led every task type: GPT-5.6 Sol led lookup, Fable 5.1 led planning, Opus 5.5 and Fable 5.1 tied on comparison, and GPT-6 Astra and Sonnet 5.5 tied on customized deliverables.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.