AA-PersonalityBench Measures Model Personality and Style at Scale

Artificial Analysis built AA-PersonalityBench to capture something accuracy benchmarks miss: two models can both answer correctly and still read very differently. Its key design choice is that it measures how strongly a trait shows up in a model's answers, not whether that trait is desirable. The launch article says traits can be positive, neutral or negative depending on what the user wants. So the lists in the announcement name the models that show the most of each quality. That is why Llama 4 Maverick, GPT-4o and K2 Horizon top the Slop index. They produced the most slop. They did not do best at avoiding it.

The personality half works by comparison rather than by a fixed checklist. The benchmark has 30 traits, such as eloquent, pragmatic and sycophantic, and 30 questions for each, so 900 in total. Every model answers all of them once, with no system prompt, so the scores describe its default behavior. A judge model from a panel then sees two anonymized answers to the same question, presented in random order and without any reasoning text, and picks the one that shows the trait more strongly, or calls a tie. Those verdicts become Elo ratings, the same win-loss scoring used for chess players. The trait ratings are standardized and averaged into eight indexes, such as Warmth, Conviction and Imagination, which are then ranked as percentiles. Artificial Analysis publishes confidence intervals and notes that models whose intervals overlap are not definitively different. It also keeps the question set private, publishing only two sample questions per trait, to avoid contamination, meaning models being trained on the test.

The communication half is simply counted. Twenty-one measurable habits, grouped into length, structure, punctuation and emphasis, are computed directly from every response. The announcement reports that Qwen3.8 27B averages about 1,030 words per response, nearly four times GPT-6 Luna's 270. Nemotron 3 Ultra averages nearly three tables per response. Z.ai's GLM-5.3 averages seven em dashes, while Anthropic's Claude Opus, Sonnet and Haiku 5.5 almost never use one. Because these figures are counts, they are more objective than the judged traits, which rest on LLM judges whose agreement the article does not report.