← All IntelClip / OtherCMU/GitHub study: AI coding tools cause temporary productivity spike, then decline
From Guide, Verify, Solve — Anirban Chatterjee, Sonar · ≈1:39
“They found that there was, in fact, a temporary spike in productivity, um but it lasted about 3 months and then it went back down.”
“And the reason for that, we think, is because there was also a persistent increase in static analysis warnings and code complexity.”
What’s in it
- Cites Carnegie Mellon data on AI coding tools' real productivity impact
- Explains why AI-assisted coding gains fade after three months
- Breaks down SonarQube findings on code quality vs AI speed
Clip transcript
get there is what we're going to talk about today. And uh for those of you who were in uh Tarik's keynote yesterday, uh he presented some of this data and I'm going to talk about it a little bit deeper today. So there was a study that Carnegie Mellon did um where they actually looked at projects that were posted on GitHub. And they were able to use the metadata to support to sort them into projects where uh there were just traditional tools that were being used and projects where an AI tool was used to write the code. In this case, it was Cursor, although it could have been any AI tool. And what they found was interesting. They found that there was, in fact, a temporary spike in productivity, um but it lasted about 3 months and then it went back down. And the reason for that, we think, is because there was also a persistent increase in static analysis warnings and code complexity. They're actually using SonarQube uh to actually collect the data on this and they saw that there was a persistent increase in these types of issues that went beyond the 3-month mark and persisted well into the future. And so, it's these types of issues that end up actually slowing developers down even more and and making And this is what makes it a challenge to deliver high-quality code using AI tools.
Comments
Sign in to comment.
Loading comments…