Intel / article

A Framework for Identifying, Categorizing, and Explaining Bias in AI-Generated Code

Source: arxiv.org
Author: Manaal Basha, Aimee M. Ribeiro, Gema Rodriguez-Perez
Date: Published Sep 28, 2026

Context

AI can help flag bias in generated code, but its usefulness varies by the kind of bias. In a preprint studying human-labeled Python snippets, researchers supplied models with bias categories and examples, then asked them to identify and explain problems. For stereotype bias, Google's Gemini 2.5 Pro identified 96.6% of the labeled cases, while 77.8% of its stereotype flags matched the reference labels. Performance was much weaker on proxy bias, where an apparently neutral attribute stands in for a protected one; that category had only five labeled examples, which the authors say limits how far its results can be trusted. The labels came from two of the authors, so the results support assistance with review while showing why aggregate scores can conceal weak categories.

Key takeaways · AI-distilled

The authors extended a dataset of biased AI-generated Python code with manual bias-category labels and human-written justifications, then used it as ground truth to test LLMs as bias detectors via in- learning.
Gemini reached 80.14% classification accuracy with 84.0% precision and 95.7% recall; the best open model, Qwen3-coder, reached 82.45% accuracy but only 68.64% precision and 80.22% recall.
Model-written explanations scored about 80% similarity to the human justifications, and code-identification similarity ran 86% to 88%, which the authors read as substantial with expert reasoning.

Terms in this piece · Glossary

context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.

Read the source arxiv.org