Vibeleaderboard
← All Intel
Intel / post

Gemini Decision 1 debuts: strong reasoning, 3x leader's cost per task

Source
x.com
Date
Artificial Analysis@ArtificialAnlys

Google has released its first decision model, Gemini Decision 1, scoring 66 on the Artificial Analysis Decision Index and leading in knowledge and reasoning benchmarks Gemini Decision 1 from @GoogleDeepMind scores 66 on the Artificial Analysis Decision Index v2.1, in line with Elephant 9B (@baselabs) and Jev 1.14 (@typesafeai), and 7 points behind leaders Drex Max v1.1 (@NaceAI) and GLiDE v1.2 (@fastinoAI), which both score 73. It is priced at $0.10 per 1M input tokens and costs $345 per 1M Tasks across the Index, around three times higher than Drex Max v1.1 ($103) due to its higher pricing Key results for Gemini Decision 1: ➤ Not yet frontier on Decision Index: Gemini Decision 1 scores 66 on the Artificial Analysis Decision Index. It leads knowledge and reasoning benchmarks, scoring 66 on AA-Omniscience-Decision and 49 on HLE-MultiChoice. It trails on other evaluations, including our new proprietary classification and extraction benchmark AA-Paperclip, where it answers all 15 questions correctly on 33% of documents vs. 63% for Drex Max v1.1 ➤ More than 3x the Cost per Task of leading models: Gemini Decision 1 costs $345 per 1M Tasks, compared with $103 for Drex Max v1.1, $156…

Read the full post on X
Why it matters

Gives pricing and benchmark positioning for Google's first decision model: it leads on knowledge and reasoning and is well , but costs about 3x more per task than the leader. Useful when picking models for classification and decision workloads.

Key takeaways · AI-distilled
  • Artificial Analysis scores Gemini Decision 1 at 66 on Decision Index v2.1, level with Elephant 9B and Jev 1.14 and 7 points behind Drex Max v1.1 and GLiDE v1.2 at 73, while it leads AA-Omniscience-Decision (66) and HLE-MultiChoice (49).
  • On AA-Paperclip, Artificial Analysis's new classification and extraction , Gemini Decision 1 answers all 15 questions correctly on 33% of documents, against 63% for Drex Max v1.1.
  • The cost gap comes from price: $0.10 per 1M input tokens versus $0.03 for Drex Max v1.1, giving $345 per 1M tasks versus $103. That is close to GPT-6 Luna (Decisions) at $339.
  • Its Brier score (lower is better) is 0.18 across the Index, between Drex Max v1.1 at 0.16 and Jev 1.14 at 0.19. It is among the lowest of all models on HLE-MultiChoice at 0.20 but trails on When2Call, 0.14 versus 0.04 for Drex Max v1.1.
Terms in this piece · Glossary
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
More from Artificial Analysis
Recommended reads
Comments

Checking sign-in…

Loading comments…