Astra 6.2 Tops Multi-Turn Vibe Code Bench
- Source
- x.com
- Date
Astra 6.2 is the #1 model on our multi-turn coding eval: Vibe Code Bench 1→100. Other benchmarks test a model’s ability to handle a single prompt. This benchmark, created by Vals, tests the ability to modify an evolving codebase over many user requests while preserving existing functionality without introducing regressions.

Context
Vals AI, a company that runs independent AI benchmarks, says OpenAI's GPT-6.2 Astra is now the top model on Vibe Code Bench 1→100, a test of whether a coding model can keep changing a working app without breaking what already works. On Vals's results page, Astra scored 55.16%, ahead of Google's Gemini 4 Carbon at 49.73% and OpenAI's cheaper GPT-6.2 Sol at 40.94%.
Most coding benchmarks hand a model one prompt and grade what comes back. Here each of 50 test tasks starts from a complete web app that passes its tests, then delivers up to ten product requests in plain English, one at a time, such as adding a feature, applying one rule across the whole app, fixing a bug, migrating the database schema or reverting an earlier change. Later requests build on earlier ones. After each request, automated browser tests check the new feature and re-check older behavior, and a model that adds the feature but breaks something else fails that step. A task's score is the share of its planned steps passed in a row before the first failure, averaged across the 50 tasks. Astra's 55.16% therefore means it got, on average, a little over halfway through each sequence of requests before a step failed or the run stopped.
These are long, expensive runs, and Vals counts the total generation cost per task: $45.66 for Astra, $57.92 for Gemini 4 Carbon and $12.74 for GPT-6.2 Sol. The chart in Vals's post marks the cost-versus-accuracy frontier, the models that no other model beats on both score and cost at once. Above two cheap Xiaomi MiMo models it runs through GPT-6.2 Sol and Astra, which leaves Gemini 4 Carbon off it: Astra scores higher for less. Anthropic's best entry, Claude Opus 5.5, scored 30.95% at $115.10, and Vals notes it fell back to other models on 12 of the 50 tasks. All results come from one version of the run in Vals's own modified OpenHands coding environment, with AI agents running the browser checks and reviewing failed verdicts, so they show how models fare in that setup rather than in every coding tool.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Checking sign-in…
Loading comments…






