Vibeleaderboard
← All Intel
Intel / article

Acquiring and Verifying Repository Norms for Coding Agents

Source
arxiv.org
Author
Kaifeng He, Xiaojun Zhang, Zhenxi Chen, Christoph Treude, Mingwei Liu
Date

Context

A code change can pass a project's tests and still break its contribution rules, which are scattered across files and history and often come with conditions and exceptions. The RepoNorm paper, submitted October 6, 2026, tries to collect those norms from the repository, check them against evidence including history, and hand them to an existing as a package.

On 121 tasks from its RepoNormBench, three coding models improved their overall norm compliance rate by 7.42 to 10.77 percent relative to a no-guidance baseline, and scored higher on these measures than with generated documentation. Functional success rose by 5.79 to 9.09 percentage points, but the paired tests did not reach significance. The authors report 86 percent sampled precision for the extracted norms.

Only the abstract was available, so the design and how the benchmark was built are not examined here.

Key takeaways · AI-distilled
  • RepoNorm extracts norms independently of any coding task, checks each norm's content and applicability against repository evidence, consults Git history when needed, and hands the result to existing coding agents as norm packages.
  • On 121 RepoNormBench tasks, the Overall Norm Compliance Rate rose 7.42-10.77% and the -omitted NCR rose 11.34-17.69% (relative) over agents given no generated guidance.
  • All three coding models scored higher on every norm-compliance measure with RepoNorm than with CodeWiki-generated documentation.
  • Functional success rose 5.79-9.09 points over the baseline, but the authors report those paired comparisons are not statistically significant; sampled norm precision was 86% in the default configuration.
Terms in this piece · Glossary
  • Git and version control — A system for recording and comparing changes to a codebase.
  • Agent mode — A mode in which one agent can inspect, edit, run commands, and iterate toward an outcome.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
  • Prompt — Information or instructions supplied to a model; in a chat, the request you type is often called your prompt.
Recommended reads
Comments

Checking sign-in…

Loading comments…