Intel / article

When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

Source: arxiv.org
Author: Anubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
Date: Published Sep 22, 2026

From the Daily Brief

The UBC-led paper generated more than 35,000 programs under 18 demographic personas and found the persona surfaced in up to 65% of responses and 70% of reasoning traces. The authors report measurable effects on code correctness, maintainability, and security, even though the persona was irrelevant to the task. A separate audit from Los Alamos National Lab ran 30.73 million generations to separate genuine identity effects from ordinary prompt noise. It tested gender, country, and occupation, and found occupation was the most consistent source of measurable differences in style and complexity. Read together, the two studies suggest that coding assistants which personalize to a user profile can change the code they return, not only the tone. Teams that inject user profile data into coding prompts should test whether outputs shift when only that profile changes.

Read the 2026-09-23 Brief →

Context

Adding a user's demographic profile was associated with different coding-test results in a study of Google's Gemini 2.5 Pro and OpenAI's GPT-OSS-120B. Researchers compared 18 profiles (combinations of nationality, gender and experience level) with a neutral prompt on the same programming problems. Gemini's average correctness fell by 1.54 percentage points, with only one profile staying significant after statistical correction, while GPT-OSS's rose by 4.52 points. The paper therefore reports sensitivity to personal , with different directions across the two models.

The authors deliberately chose the prompt format that most reliably elicited demographic references, which they acknowledge may amplify the effect relative to real use. The findings are framed as associations. The result supports testing personalization as part of a coding assistant's behavior, rather than assuming irrelevant user details leave technical answers unchanged.

Key takeaways · AI-distilled

The study compared a proprietary model (Gemini 2.5 Pro) with an one (GPT-OSS-120B), prompting each with 18 personas spanning nationality, gender and experience level against a neutral baseline.
On LiveCodeBench, persona prompts lowered Gemini's correctness by an average of 1.54 percentage points, with one persona down 3.6% (odds ratio 0.51).
The direction differed by model: GPT-OSS accuracy rose 3.4 to 5.7% across every persona (odds ratios 1.8 to 3.0), so persona effects are not a uniform penalty.
Maintainability and style differences were statistically significant but negligible (Cliff's delta below 0.15), and the authors found no systematic persona-specific pattern in security vulnerabilities.

Terms in this piece · Glossary

context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.

Read the source arxiv.org