Intel / article

Methodological Harness in Agentic Software Engineering: An Empirical Study on Mining Software Repositories

Source: arxiv.org
Author: Jessica D\'iaz, Jorge P\'erez, Sergio Gil-Borr\'as
Date: Published Sep 29, 2026

Context

Visible instructions for coding agents are more common than integrated processes for governing their work, according to this repository study. Researchers analyzed 5,435 repositories sampled from a collection of 116,211 GitHub repositories with activity, stratified by popularity measured in stars. They estimated that 21.7% of the population contained at least one observable mechanism, compared with 65.3% among the most visible repositories. Multi-mechanism configurations were rare; where rule files exist, they almost always guide the agent and state norms. The result shows why examining only prominent projects can exaggerate apparent adoption. It measures repository artifacts, without establishing that those artifacts improved software quality or governed what agents actually did.

Key takeaways · AI-distilled

Researchers checked whether an eight-mechanism methodological for agentic development, from to graduated autonomy, shows up in practice across 5,435 repositories sampled from the 116,211-repo AIDev dataset.
At least one mechanism appears in 21.7% of the population but in 65.3% of the most visible repositories, measured by stars. Repositories combining several mechanisms are rare.
Coding rule files from 150 repositories showed they almost always guide the agent and state norms, while shared persistent knowledge, executable specifications, structured consultation and graduated autonomy appear only in a minority.
The authors conclude the harness is observable but mostly as isolated mechanisms, not the integrated system the framework proposes.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
context engineering — Deciding what an agent sees at each step — the discipline that replaced prompt engineering once context windows got large enough to fill badly.

Read the source arxiv.org