FairStress: LLMs drop correct decisions asymmetrically by demographic

Most fairness tests for language models check one thing: whether a model makes the same choice when a candidate's demographic details are swapped. Ai2's FairStress asks a harder question. When a model has already made a correct, well-supported decision, how easily can it be argued out of it, and does that depend on which group the decision favors? The benchmark pairs 300 two-candidate decisions across 29 high-stakes domains with what Ai2 calls adversarial prefixes, single persuasive sentences that appeal to authority, social consensus or purported evidence on behalf of the weaker candidate (GitHub, Ai2 blog). Varying how explicitly identity is signaled and which groups are contrasted expands this into more than 13 million test cases.

The repository defines four measurements. Accuracy Gap asks whether the model is right equally often whichever group the right answer favors. Robustness Gap asks whether a correct answer is abandoned equally often under a prefix. Two tie-break measures look at cases where neither candidate is better: how often the model picks the minority-coded candidate, where 50% is fair, and whether a prefix reverses its choice equally often for both groups. A fair model scores zero on the gaps.

Ai2 reports that asymmetry was small when no demographic information was given and grew as identity became more explicit, in a consistent direction: models were more willing to drop a correct choice when the switch favored the minority-coded candidate. Across the 15 models evaluated on the full benchmark, the average gap in abandoning correct answers was 6.9 percentage points with implied identity and 9.3 with explicit labels. On ties, Ai2 says that on FairStress-Core, a 50,000-item subset of the benchmark, frontier models including Anthropic's Claude Opus 5 and OpenAI's GPT-5.5 picked the minority-coded candidate about 60 to 65% of the time. Ai2 calls this anti-stereotypical over-correction, which is the opposite of the stereotype-following bias older benchmarks usually look for. It also reports that across 11 open-weight models, explanations for minority-coded choices used somewhat more confident and positive language, and when identity was explicitly labeled included words like resilience and diversity more often, while explanations for majority-coded choices leaned on procedural terms such as evidence and criteria.

Fixing the gap proved hard. Of nine post-hoc debiasing methods Ai2 tested, most did not reliably reduce the asymmetry and some cost substantial accuracy. Counterfactual chain-of-thought came closest, narrowing the gap while largely keeping accuracy, but did not remove it. These results come from Ai2's own blog post and code release; the full tech report and the rest of the announcement thread were not reviewed here.