When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation
Source
Anubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
Author
Anubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
Date
Key takeaways · AI-distilled
The study compared a proprietary model (Gemini 2.5 Pro) with an open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → one (GPT-OSS-120B), prompting each with 18 personas spanning nationality, gender and experience level against a neutral baseline.
On LiveCodeBench, persona prompts lowered Gemini's correctness by an average of 1.54 percentage points, with one persona down 3.6% (odds ratio 0.51).
The direction differed by model: GPT-OSS accuracy rose 3.4 to 5.7% across every persona (odds ratios 1.8 to 3.0), so persona effects are not a uniform penalty.
Maintainability and style differences were statistically significant but negligible (Cliff's delta below 0.15), and the authors found no systematic persona-specific pattern in security vulnerabilities.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
Why it matters
Shows persona or demographic cues in a coding prompt measurably change code quality and security outcomes even when irrelevant to the task, a concrete bias risk for any coding assistant that personalizes to user identity.
articleIdentity or Prompt Noise? A Calibrated Invariance Audit of LLM Code GenerationMaksim E. Eren, Ryan Barron, Eric Michalak, Charles Nicholas, Manish Bhattarai
articleA Study on the Impact of Natural Language Differences in Prompts on Automatic Code Generation Using LLMsHaruka Tokumasu, Masanari Kondo, Alexander Serebrenik, Dong Wang, Kei Koyanagi, Kotaro Noguchi, Naoyasu Ubayashi, Yasutaka Kamei
articleDo Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code GenerationAlex Deaconu, Anubhav Gupta, Manaal Basha, Nicholas Haydu, Gema Rodr\'iguez-P\'erez