Intel / article

Senior engineer benchmark: eight models, one refactor task

Source: checks.every.to
Author: Dan Shipper
Date: Published Oct 13, 2026

Context

The 21% that Dan Shipper cited for Google's Gemini 4 Argon comes from a deliberately narrow (x.com). The scoreboard describes the Senior Engineer Benchmark as one person's test: real work Shipper uses AI for, scored by the things he checks when judging whether output is good, meant to answer which model is best at his job rather than to make a general claim about model quality (checks.every.to). It currently holds one task, labeled proof-rewrite-integrated, which Shipper's Gemini 4 vibe check describes as refactoring a messy vibe-coded codebase.

Each run is scored as the share of applicable checks it passed. Some checks are answered by a program and others by an judge, which the page says is always Claude. On this task, Anthropic's Claude Opus 5.5 and Claude Sonnet 5.5, both run in Claude Code at high effort, scored 85%, followed by OpenAI's GPT-6 Astra at 81% and GPT-6.1 Sol at 71% in Codex CLI. Two Muse Wonder 1.0 variants scored 50% and 48%, GPT-6 Sol 35%, and Gemini 4 Argon, also run through Codex CLI at high effort rather than in Google's own tools, came last at 21%.

Key takeaways · AI-distilled

On Every's Senior Engineer Benchmark refactor task, Claude Opus 5.5 and Sonnet 5.5 in Claude Code scored 85%, GPT-6 Astra in Codex CLI 81%, GPT-6.1 Sol 71%, GPT-6 Sol 35% and Gemini 4 Argon 21%.
The benchmark is one person's real work scored by the checks they use to judge output, so it answers which model is best at that job rather than making a general claim about model quality.
Code checks are answered by a program and model checks by an LLM judge that is always Claude; a run scores checks passed over checks applicable, and each case counts only its latest public run.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source checks.every.to