How Reasoning Shapes Social Bias in LLM-Generated Code?
- Source
- arxiv.org
- Author
- Weifeng Sun, Jieke Shi, Zhou Yang, Yuchen Chen, Hongyan Li, Meng Yan, David Lo
- Date
Context
Having a model reason before writing code can reduce demographic bias while introducing a different quality trade-off. Across nine models and three decision scenarios, researchers report that reasoning reduced the average measured bias rate from 0.64 to 0.40, but average code quality also declined. They also found that biased reasoning strongly predicted biased programs. Research abstract
Their proposed response, ProbeDebias, checks and rewrites biased reasoning before code generation. The authors report that this reduced code bias while largely preserving quality in their . The finding makes the reasoning stage a concrete intervention point: the assumptions used to construct a program can matter alongside the program's final output. Method and reported results
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleA Framework for Identifying, Categorizing, and Explaining Bias in AI-Generated CodeManaal Basha, Aimee M. Ribeiro, Gema Rodriguez-Perez
articleWhen Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code GenerationAnubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
articleChain-of-Models: Cross-Model Auditing for Bias-Robust LLM JudgesQian Wang, Zhanzhi Lou, Zhenheng Tang, Nuo Chen, Bingsheng He
Checking sign-in…
Loading comments…