Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms
Source
Shiqi Cheng, Evelyne Ringoot, Rabab Alomairy, Alan Edelman
Author
Shiqi Cheng, Evelyne Ringoot, Rabab Alomairy, Alan Edelman
Date
Key takeaways · AI-distilled
Getting correct parallel code out of an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → took modest prompting. Getting code that actually ran faster was the hard part. Correctness and speedup are separate outcomes, and benchmarks usually measure only the first.
Across sorting, graph traversal, and search, one agent produced consistently correct parallel implementations with no measurable speedup at all, while another delivered the strongest gains. Accuracy-only scoring makes those two look identical.
No language won everywhere: C++ parallelized most reliably for graph algorithms, while Python and Julia produced the largest speedups on search. Python and Julia gained on some graph algorithms and regressed on others.
The paper's argument is that runtime performance belongs in coding benchmarks next to accuracy. For parallel work a passing test says nothing about whether parallelizing helped or made things slower.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Coding agents that look competent on serial code degrade on parallel algorithms — worth knowing before delegating that work.
Key quotes
“We find that coding agents can produce correct parallel implementations with modest prompting effort, but that achieving meaningful speedup is heavily algorithm- and language-dependent.”
“Sonnet 4.6 delivers the strongest overall performance gains, whereas GPT 5.4 produces no measurable speedups despite consistent correctness.”
“C++ is most consistently parallelizable for graph algorithms, while Python and Julia achieve the largest speedups on search algorithms: no single language dominates across all categories.”
“However, recent initial results suggest their parallel programming capabilities lag behind serial programming capabilities.”
