What’s the ROI of software engineers’ spending on AI?

In the publicly readable part of his survey, Gergely Orosz finds that engineering teams' spending on AI tools has grown large and very uneven, while its link to output is unclear. He gathered figures from engineering leaders and senior engineers at 27 workplaces and found no connection between company size and spend. A two-person .NET shop spends $3,000 per developer per month, while a publicly traded company with 1,500 engineers caps developers at $150. One quant trading firm's total AI bill went from $500K a month in January to $8M in September. At nearly all the companies, spending peaked in June and July and has roughly halved for companies paying per API call. Most companies attribute the drop to a mix of Opus 5.5 and moving to open models.

To judge the payoff, Orosz uses the change in pull requests (PRs), the unit in which developers submit code changes for review. The PR numbers don't all move in the same direction. Aggregate data from CodeRabbit, an AI code review tool, shows a 113% rise in weekly PRs among its users compared with a year ago. The two-person shop says it went from 1-2 PRs a day to 10 or more. But other companies in the free sections report flat PR frequency with PRs three times larger, or little change in velocity once a team holds code to its pre-AI quality bar. One staff engineer described seeing "accelerated slop" around him. The company accounts are mostly anonymized reports from individual engineers, and CodeRabbit's figures cover its own users rather than developers in general.

This explanation is based only on the free portion, which covers the CodeRabbit and Gitar data, the overall spending patterns and the high spenders. The sections on Big Tech, medium and low spenders, and the article's discussion of ROI are paywalled and were not read. The same newsletter previously asked whether engineering departments were trying to cut back on AI spend.