What effort really changes in Claude Code, backed by Terminal-Bench 3.0
- Source
- x.com
- Date
What is effort really? When do you change it it and why not just use max effort for everything? I dove deep into this problem, looking into evals and doing my own tests and I was quite surprised by the results. https://t.co/KO2D51j34H
Context
In an X article, Thariq, writing from Anthropic about "our newest Claude models", argues that the effort setting in Claude Code mainly changes how much verification and edge-case testing the model does and how much it relies on its own judgment. He describes effort as a rough signal of how much compute you want spent on a task, and compares it to being given 12 hours rather than 1 hour for the same job: with less time you deliver a reasonable version and expect to iterate. Claude attempts the task either way, but at higher effort it takes more independent action to make judgment calls and check its work.
His evidence for hard tasks comes from his own runs on Terminal-Bench 3.0, a community-sourced of demanding terminal tasks in areas such as security, hardware, machine learning and science. On html-js-filter, which asks for an HTML sanitizer that strips every way of smuggling JavaScript into a page, Fable 5.1 succeeded in 1 of 5 attempts at low effort and 5 of 5 at xhigh. A typical low-effort attempt took about 2 minutes, wrote the filter in roughly one pass and tested it against a single hand-written page. The high-effort run he traced took about 33 minutes: it reviewed its own first draft adversarially, read the installed parser's source for bugs, ran a standard XSS test suite and wrote a fuzzer that generates random documents. Across all the results, he says, more effort tends to cut failures caused by missed edge cases but does not fix attempts where the model chose the wrong approach.
That is why he does not recommend maximum effort everywhere. In his smaller build and design experiments on Opus 5.5, low effort returned a quick starting point, such as a 1-minute sketch of a redesigned settings menu against 28 minutes for a polished mockup at max effort, while higher effort also meant Claude made more assumptions on his behalf. His working loop for feature development is to give Claude a spec and have it interview him about missing details, implement and iterate at low effort, then verify and test at high effort. He keeps high effort for work where edge cases matter, such as fixing a bug in an existing codebase, and max for problems Claude must solve fully on its own. The results are one Anthropic author's five-attempt runs and illustrative examples, not an independent .
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Checking sign-in…
Loading comments…






