Vibeleaderboard
← All Intel
Intel / article

EverydayBench: Evaluating Satisfactory Completion of Live User-Facing Digital Work

Source
arxiv.org
Author
Fangrui Huang, Bingyang Wang, Chuhan Li, Yijiang Li, Arpandeep Khatua, Austin Shen, Riki R Z Choi, Cuiyuanxiu Chen, Jin Huang, Zachary Robertson, Kate Lin, Akshay Paruchuri, Sijun Tan, Xin Liu, Zhengmin Ni, Sanmi Koyejo, Ehsan Adeli
Date
Why it matters

Live-web agent tasks have shifting valid answers; this fixes success criteria up front and checks current evidence, giving a more trustworthy way to evaluate browsing agents.

Key takeaways · AI-distilled
  • EverydayBench has 77 human-authored tasks across eight domains of everyday life and work. Each task fixes its success criteria before execution without prescribing an answer or workflow, so agents choose their own outcomes.
  • Scoring combines three checks: deterministic tests of explicit constraints, verification of claims against the pages the cited, retrieved again at time, and a rubric for the quality of the final deliverable.
  • Across 23 agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tied for the highest strict pass rate at 54.55%. In a controlled effort sweep, GPT-6 Sol went from 29.87% at low effort to 45.45% at max effort.
  • No model led every task type: GPT-5.6 Sol led lookup, Fable 5.1 led planning, Opus 5.5 and Fable 5.1 tied on comparison, and GPT-6 Astra and Sonnet 5.5 tied on customized deliverables.
Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Recommended reads
Comments

Checking sign-in…

Loading comments…