Artificial Analysis rebuilds its medical long-context leaderboard

MLCR is Wisedocs' Medical Long Context Reasoning benchmark, and Artificial Analysis runs a leaderboard on it called MLCR-AA. Artificial Analysis has changed how that leaderboard prompts and grades models, saying it wants to better reflect how medical record review is done in practice. Models are now prompted as medical record reviewers and must answer each part of a question in one paragraph that gives the conclusion and the supporting evidence from the record. A new panel of three judge models grades the answers. Artificial Analysis also checked all 60 expert reference answers against their source records and refined 15 of them, so the prompt, the grading and the answer key all changed in this update.

The pass rule explains why the top scores are so low. An answer counts only if it is both complete, meaning it reaches the expert's conclusion with the right medical reasoning, and accurate, meaning it cites the record without exaggerating or inventing details. Anthropic's Claude Sonnet 5.5 and OpenAI's GPT-6 Astra tie for first at 41.7%, with GPT-6.1 Sol at 38.3%. The median model passes under 15% of questions. The leaders also fail in opposite ways. Claude models hold the top 18 completeness scores, led by Sonnet 5.5 at 64.8%, while the best non-Anthropic model reaches 42.2%. OpenAI models hold the top 27 accuracy scores, led by GPT-6.1 Sol at 96.7%, but Sol is complete on only 40.0% of questions. In short, Claude more often reaches the full answer, and OpenAI's models more often keep their citations faithful while leaving parts out.

The best open-weights model is Qwen3.8 2.4T A95B at 30.6%. The figures are for specific effort settings. The top Claude result is Sonnet 5.5 at max effort with fallback, and the OpenAI results are at max. Artificial Analysis does not say in the post which models make up the judge panel.