Vibeleaderboard
Index / tool
Visit github.com
Category
Developer Tools
Rank
No. 1991Tools index

Previous survey · No. 1929 ·

Pricing
Open Source
Type
TOOL
Use case
Model & Agent Evaluation
GitHub
4 stars
Date

About

jev-effort is a research project testing whether letting Jev, TypeSafe's small decision model, choose Claude Code's reasoning effort on a per-step basis reduces API costs without breaking the prompt cache. At maximum effort it cut the cost of a coding benchmark by 55% while every test still passed, and in a long real session it saved an estimated 4-9%, since re-reading context accounts for most of the bill.

At a glance

jev-effort proxies Claude Code requests and asks a second model how much reasoning effort to use. It is worth reading as an experiment if you pay for high-effort sessions and can measure whether lower effort changes the work.

Good fit

Developers comparing cost and task success on disposable coding tasks with known tests.

Consider something else when

Keep a direct connection if prompts or tool outputs cannot go to a second provider.

Start here

Read the shadow-mode results first. If evaluating it yourself, use a synthetic task, compare equivalent runs and count total cost, test outcomes and any extra turns.

Cost and requirements

MIT-licensed. The author documents Node 20 or newer, Claude Code, Claude access and a Jev key.

Limits to know

Jev receives a trimmed conversation that can include code and tool output. The author’s real-session savings are estimates from shadow mode, which does not show whether the lower-effort run would have succeeded.

Tags

claude-codereasoning-effortbenchmarkllm-costresearch

Tech Stack

Node.js

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.