Software factories and code reviews with Harjot Gill
- Source
- newsletter.pragmaticengineer.com
- Author
- Gergely Orosz
- Date

Context
Harjot Gill, founder and CEO of the AI code review company CodeRabbit, says he shipped more than 5,000 pull requests in a month by running coding agents, at about $150 each. A pull request is a proposed change to a codebase that normally waits for a colleague to read it before it merges. At that volume nobody can read every change, so his conversation with Gergely Orosz on The Pragmatic Engineer podcast is really about what replaces that reading. According to Orosz's write-up of the episode, Gill's answer is to move checking out of people's heads and into machine checks the agents can run on their own work, and to move human review up to plans and whole releases.
Gill's term is that agents must be able to hill climb: they need a way to test their output and keep improving against that test. He lists layers that do this. Formal models, written in languages such as TLA+ or Lean, describe a design precisely enough for software to check it; a TLA+ model at CodeRabbit found race conditions, bugs that appear only when two operations happen in an unlucky order, that code review had missed. Local simulations replay many orderings of concurrent operations deterministically. Emulated versions of outside services such as Google Cloud APIs let tests run without the real thing. Agents also deploy to development environments to confirm services work. In production, features ship behind flags and roll out in stages, with an watching performance and logs before widening the release.
People still review, but at a different level. Orosz reports that CodeRabbit engineers review releases made of many pull requests rather than individual ones, and that some engineers are on a list allowed to push to production without review. When an outage is traced to someone's change, that person gets a strike, and after enough strikes their releases need a colleague's review again. Orosz likes the pairing of trust with accountability.
He is also clear that this is an outlier. Gill puts his own agent spending at roughly $250,000 to $275,000 a month, while Orosz says the figures he hears elsewhere run from about $200 to $2,000 per developer per month. Orosz's view is that, at a developer-tools startup that wins by shipping new products quickly, the spending can work like research and development if it produces products CodeRabbit can sell, and that the case is different for a company whose customers will pay only a set amount for an existing product. These details come from Orosz's written summary and show notes; the full episode transcript was not available for this explanation.
- CodeRabbit CEO Harjot Gill says he shipped over 5,000 pull requests in a month with agents, much of it janitorial work such as CI improvements, flaky-test fixes and caching, plus larger builds like a Rust write-ahead log on object storage that one agent worked on for six weeks.
- Gill puts his personal AI spend at about $250-275K a month, roughly $150 per pull request. Host Gergely Orosz calls this an outlier: the per-developer AI spend he hears about elsewhere runs between $200 and $2,000 a month.
- Quoted $13M a year for a CI system that could handle its pull request volume, CodeRabbit had agents build its own instead. Gill says it took a week and costs 50% less than the quote with similar performance.
- Gill wants agents to verify their own work in layers: TLA+ and Lean formal models (TLA+ caught race conditions code review missed), deterministic simulations that fuzz concurrent interleavings, emulated cloud APIs, real dev deployments, and staged rollouts watched by an agent.
- At CodeRabbit, engineers review releases spanning several pull requests rather than individual ones, and some may ship without review at all. A developer whose change causes an outage gets a strike, and enough strikes put their releases back under colleague review.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Checking sign-in…
Loading comments…





