HLE-Diamond leaderboard: reasoning outpaces knowledge, effort costs add up

HLE-Diamond is a cleaned-up version of Humanity's Last Exam, a benchmark of closed-ended questions, and Artificial Analysis now runs it as part of its Intelligence Index v5, where it carries 10% of the weight. According to the Artificial Analysis thread, the Center for AI Safety and Scale AI released it after a year of refining the original dataset with input from researchers. It has 1,000 questions, half testing reasoning and half testing knowledge, and 272 of them include an image.

The scoring method shapes what the numbers mean. Artificial Analysis asks each question once, with no tools, using the official HLE system prompt. A separate model, GPT-5.6 Luna at medium effort, compares each answer with the reference answer using the official HLE judge prompt, and the score is the share judged correct on that single attempt. A model therefore gets no retries and cannot look anything up, so the knowledge half tests what the model already holds.

Under those conditions, only 17 model variants, about 13% of those tested, score above 50%. Gemini 4 Carbon at high effort leads with 69%, one point ahead of Claude Opus 5.5 at max effort, which led this benchmark when Index v5 launched a few days earlier. More than 90% of the models measured do better on reasoning than on knowledge. Claude Opus 5.5 and GPT-6 Astra both score 78% on reasoning, but Opus 5.5 scores 58% on knowledge against GPT-6 Astra's 50%. Gemini 4 Carbon leads both halves, with 80% on reasoning and 59% on knowledge.

Open-weights models are far behind. The best is Kimi K3.1 Pro at 37%, and its weights have not been released yet. MiniMax-M3.1-Flash follows at 28% and Kimi K3 at 27%. The Index v5 announcement found the same pattern: HLE-Diamond is one of the two benchmarks where the gap between open-weights and proprietary models is widest.