Vibeleaderboard
← All Intel
Intel / post

Vals AI Finds Devin Fusion on the Pareto Frontier for Code Migration

Source
x.com
Date
Vals AI@ValsAI

We ran Devin Fusion on our Code Migration benchmark and the performance of Devin Fusion with GPT-6 Astra and SWE-2 sidekick places it on the Pareto Frontier of the benchmark. *Code Migration Bench is our benchmark that determines if models can reimplement working programs in another language.

Context

Vals AI ran Cognition's Devin Fusion, which pairs a frontier model with a cheaper sidekick model on some steps, against its Code Migration Bench, a testing whether an can correctly reimplement a working program in a different language. Vals AI's post says the combination of Devin Fusion with GPT-6 Astra and the SWE-2 sidekick model lands on the benchmark's Pareto frontier, meaning no other tested setup was both cheaper and more accurate on this task. The post does not include the underlying accuracy and cost figures, so the size of that advantage over the next-best option is not established here. Artificial Analysis separately benchmarked Devin Fusion on a different task set, finding its GPT-6 Astra pairing ran cheaper and faster than a Claude Fable 5.1 pairing while scoring close behind it, a similar cost-versus-accuracy tradeoff on a separate benchmark.

Terms in this piece · Glossary
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
More from Vals AI
Recommended reads
Comments

Checking sign-in…

Loading comments…