agent-roast

At a glance

agent-roast is a free, MIT-licensed command-line tool that compares the before and after of a code change. It flags edits that make a failing build pass by weakening a check instead of fixing the code, such as a loosened assertion, a skipped or deleted test, a lowered coverage threshold or a CI step allowed to fail. It runs offline and never calls a model.

Good fit

It suits teams that let Claude Code, Codex or Cursor edit code and want a deterministic tripwire when an agent turns tests green by weakening them. It can run as an agent hook, a Git pre-commit hook or a GitHub pull request check. Its rules come from the base branch, so a change under review cannot switch them off.

Consider something else when

Skip it if you want proof that code is correct or a check of who wrote a change. It reads text, not program meaning, so it only notices when a check gets weaker. The full before and after comparison of test code covers JavaScript, TypeScript, Python and Go. For Rust, Java, Kotlin, C#, Ruby, PHP and Swift, test files are checked only for skipped tests and placeholder assertions, though CI and config files are checked in any language.

Start here

Run npx agent-roast demo. It reviews a scripted example in a temporary repository and reads none of your files. Then run npx agent-roast replay to see what it would have said about your last 50 commits. If the results look useful, npx agent-roast hook install adds the hook for Claude Code, Codex and Cursor.

Cost and requirements

The tool is free under the MIT licence and uses no API tokens, network access or telemetry. You need a Git repository and a way to run npx, which implies Node.js. Pull request checks use a GitHub Action (Nari-Ab/agent-roast@v1) with a full-history checkout. Rules, severities and ignored paths go in .agentroastrc.json, and npx agent-roast init writes a starting file.

Limits to know

The project's own benchmark of 64 cheating edits and 44 harmless look-alikes was written alongside the detector, so it guards against regressions rather than proving accuracy. Three known gaps remain open. About one finding in five was a false alarm in the author's sample, and one AI assistant labelled those results. The agent hooks were tested with simulated input, not long live sessions.

Still unclear

• Which Node.js versions does the npx command support?