Intel / article

Does AI-Assisted Development Homogenize Software? An Empirical Study of Code Similarity, Quality, and Evolution

Source: arxiv.org
Author: Luciano Marchezan, Romain Lefeuvre, Kevin Delcourt, Benoit Combemale, Houari Sahraoui
Date: Published Oct 20, 2026

Context

This study looked for signs that coding with LLMs and agents makes software more alike or worse, and in 50 long-running open-source projects it mostly did not find them. The researchers took code snapshots of 25 Java and 25 Python projects from three periods: before ChatGPT's release, after it (when help was possible but unconfirmed), and once traces of coding agents appeared in a project, detected through 196 heuristics on things like configuration files and commit authorship. They measured how similar the code was, both in meaning using and in structure, along with standard quality metrics from SonarQube and how fast each codebase grew.

Similarity did not rise. In Java it fell significantly in both the LLM and periods, and Python showed no significant change. Quality held broadly steady, with the agent period showing about 7% lower code-smell density and 11% lower technical-debt density than the pre-LLM period. Growth in files and other structure was generally lower in the agent period, while Java commits and added lines rose, though those activity increases were not statistically significant.

The authors are careful about what this means. The design is observational, with no comparison group of projects that never adopted AI, so project aging or changing practices could explain the shifts, and an agent-era label does not show how much of a snapshot an agent wrote. Their conclusion is that AI-assisted development does not necessarily homogenize code or degrade quality, not that it improves it, and they ask that non-significant results be read as possible trends rather than proof of no effect. The preprint covers only mature projects meeting its selection criteria.

Key takeaways · AI-distilled

The study tracks 50 long-lived Java and Python open-source projects across three eras (pre-LLM, potentially LLM-assisted, detected agentic), using Software Heritage snapshots.
Similarity was measured two ways: embedding-based cosine similarity for semantics and CodeBLEU* for syntax; quality used SonarQube maintainability, complexity, reliability and security metrics.
Contrary to the homogenization hypothesis, Java code became significantly less similar in both the LLM and agentic eras versus pre-LLM, while Python showed no significant change, the authors report.
Quality stayed largely stable, though the agentic era showed significantly lower code-smell and SQALE index densities, the authors report.
Structural growth, especially in files, was generally lower in the agentic era, while Java development activity rose moderately, especially in commits and added lines, per the study.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.

Read the source arxiv.org