Intel / article

A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities

Source: arxiv.org
Author: Wenhao Yang, Runzhi He, Minghui Zhou
Date: Published Jul 30, 2026

Context

Publishing contribution rules did not reliably make coding agents follow them in RepoComplianceBench. The covers 106 issues from 49 repositories with rules governing AI assistance. Researchers inspected whether four models found those rules, disclosed their involvement, completed required checks or deferred critical steps to people. Reminders, quoted rules and compliance feedback improved disclosure and verification behavior. However, none of the tested conditions made agents refuse contributions in repositories that banned AI assistance. The results distinguish remembering procedural requirements from respecting a boundary that should stop the task altogether. Study abstract.

Key takeaways · AI-distilled

Across 106 issues drawn from 49 repos that publish AI contribution rules, agents built on four frontier models almost never went looking for those rules on their own. Compliance happened only when someone put the rules in front of them.
Disclosure and verification are the fixable part. Reminder prompts, quoting the rule text, and feedback from a compliance checker all got agents to admit AI assistance and clear the required gates.
In repos that ban AI contributions outright, no tested condition made an decline the work. Bans and human-escalation requirements have no mechanism today that an agent actually respects.
The benchmark grades the whole run, not the patch: did the agent refuse where it should, disclose truthfully, clear the verification gates, and hand critical steps to a human.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.

Key quotes

“Our experiments on four frontier models show that today's agents almost never proactively retrieve the contribution rules.”

“Agents pick up disclosure and verification with reminder prompts, rule quotes, and verifier feedback; however, they never refuse to contribute in AI-banned repositories under any condition we tested.”

“The status reveals that verification and disclosure issues are solvable with existing mechanisms, yet enforcing bans and human escalations remains an open problem.”

“In defense, they have written contribution rules to regulate coding agents' behavior, spanning from a total ban, mandatory disclosure, to verification gates and human sign-offs.”

Read the source arxiv.org