Vibeleaderboard
← All Intel
Intel / article

Documentation Drift in AI-Generated Pull Requests

Source
arxiv.org
Author
Will De Michele, Kla Tantithamthavorn
Date
Why it matters

Coding agents skip documentation in most PRs, and undocumented changes are later fixed more often. Add doc-update checks to PR workflows.

Key takeaways · AI-distilled
  • Across 29,669 AI-generated pull requests in 2,585 repositories, the authors report that 63.2% changed code without updating any documentation file, rising from 55.2% for features to 77.5% for refactors.
  • Undocumented code files were followed by a fix to the same file 8.6-11.7% more often over 30-90 days. A naive PR-level comparison shows the reverse, because documented PRs are 3.9x larger; controlling for size flips the result.
  • Documentation debt compounds, per the abstract: within 90 days, files changed without documentation were later re-documented at about half the rate of documented ones (24% vs 48%).
Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Recommended reads
Comments

Checking sign-in…

Loading comments…