GPT-6 Sol and Luna halve costs, with mixed benchmark results
- Source
- x.com
- Date
GPT-6 Sol and Luna push the cost efficiency frontier by halving cost relative to GPT-5.6 Sol and Luna. Intelligence Index and Coding Agent Index scores remain level with GPT-5.6, with progress in some evaluations and regressions in others Pricing is approximately half that of GPT-5.6: Sol drops from $4/$20 to $2/$10 per million input/output tokens, and Luna from $0.20/$1.20 to $0.10/$0.50, with the same 90% discount for cache reads and 25% premium for cache writes. Key takeaways: ➤ Halves Cost per Task: GPT-6 Sol (max) costs $1.06 per task to run the Artificial Analysis Intelligence Index, ~50% less than GPT-5.6 Sol (max) at $1.99. GPT-6 Luna (max) costs $0.07 per task, ~60% less than GPT-5.6 Luna (max) at $0.18. This is driven by the price cut, as both models use slightly more output tokens per task (31k vs 29k for Sol, and 51k vs 41k for Luna). These two releases allow OpenAI to capture a significant portion of the cost efficiency Pareto frontier. ➤ In the Coding Agent Index, Sol improves but Luna regresses: In OpenAI's Codex harness, GPT-6 Sol (max) scores 57 in the Artificial Analysis Coding Agent Index, up 2 points from GPT-5.6 Sol (max), with gains in Terminal-Bench 4.0…
Context
Artificial Analysis, an independent firm that runs its own benchmark suites on new models, reports that OpenAI's GPT-6 Sol and GPT-6 Luna cost about half as much per task as the GPT-5.6 versions they replace while scoring level with them on its Intelligence Index and Coding Index (Artificial Analysis on X). The saving comes from price alone. Sol drops from $4 and $20 per million input and output to $2 and $10, and Luna from $0.20 and $1.20 to $0.10 and $0.50, while both models write slightly more output per task: 31,000 tokens against 29,000 for Sol, and 51,000 against 41,000 for Luna. Running the Intelligence Index costs $1.06 per task on Sol at max effort, down from $1.99, and $0.07 on Luna, down from $0.18.
Level overall scores hide movement in both directions. Run in OpenAI's own Codex , Sol gained 2 points to 57 on the Coding Agent Index, with Terminal-Bench 4.0 rising from 37% to 43%, while Luna lost 2 points to 41. The large drop in needs careful reading. On AA-Omniscience, the firm's knowledge and hallucination test, Sol's hallucination rate fell from 92% to 60%, and the firm says it got there by declining to answer more often: it attempted 83% of questions instead of 99%. That cut its wrong answers by about a quarter, but it also lowered accuracy from 59% to 54%. Luna's accuracy stayed roughly flat at 44% against 43%.
The clearest losses are in office-style work. On GDPval-AA v2.1, which grades deliverables for economically valuable tasks across 44 occupations, Sol fell about 100 rating points and Luna about 75. Luna also fell about 45 points on AA-Briefcase v1.1, a private test built on multi-week knowledge-work projects, where Sol held level. Artificial Analysis says its team inspected hundreds of outputs and found the drops came mostly from weaker presentation and from deliverables that left out items the grading rubric asks for. Overall, the new models score about the same as their predecessors for half the money, but on these tests their finished reports and documents are graded as less complete.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
Checking sign-in…
Loading comments…