AA-PersonalityBench ranks frontier models by sycophancy

Artificial Analysis has published the Sycophantic results from AA-PersonalityBench, its new benchmark of how models write rather than whether they are right. The trait asks whether a model excessively flatters or agrees with the user rather than responding honestly. Its chart places GPT-4o, Llama 4 Maverick and GPT-3.5 Turbo at the top, which the firm notes are the three oldest models it measured. Grok 4.7, Claude Fable 5.1 and Inkling sit at the bottom.

The numbers are ranks, not rates. According to the launch article, each of the benchmark's 30 traits gets 30 dedicated questions. Some of them gently invite the behavior, and each is sent as a single message with no system prompt, so the test captures a model's default style. LLM judges from a panel then compare two models' answers to the same question without seeing model names and pick which shows more of the trait. Those verdicts become Elo-style scores, which are turned into a percentile from 1 to 100 across every model tested. A 97 therefore means a model looked more sycophantic than almost everything else in this set, not that 97% of its answers flattered the user. The chart's error bars are wide for many mid-table models, and the article says models whose confidence intervals overlap are not definitively different.

The post highlights a pattern by lab. All five Meta models measured rank 65 or higher, led by Llama 4 Maverick at 97 and Muse Spark 1.3 at 87. Google DeepMind's models are spread out, from Gemini 3.1 Pro Preview at 89 and Gemini 4 Argon at 81 down to Gemini 3.8 Flash at 28. Every Anthropic model measured ranks 55 or below: Haiku 5.5 at 55, Opus 5.5 at 46, Sonnet 5.5 at 30 on medium effort and 25 on max, and Fable 5.1 at 4. Grok 4.7 is lowest of all, at 1 on the chart.

Sycophancy is one of 30 traits in the benchmark, which Artificial Analysis announced two days earlier along with eight broader personality indexes. The article says traits are measured for how strongly they show, not whether they are desirable. In one place it does count against a model: it is inverted inside the Transparency index, so a more sycophantic model scores lower there. The result is a judgment by AI judges on a private question set, so it shows how these models compare under one method, not a settled measure of how often each will tell a user what they want to hear.