Intel / article

Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms

Source: arxiv.org
Author: Shiqi Cheng, Evelyne Ringoot, Rabab Alomairy, Alan Edelman
Date: Published Jul 30, 2026

Context

An can produce correct parallel code without making a program faster. Researchers asked three coding agents to convert sequential implementations of sorting, graph traversal and search into parallel versions in C++, Python and Julia. Correct implementations were achievable with modest prompting, but speed improvements depended on the algorithm and language. Claude Sonnet 4.6 delivered the strongest overall gains in this , while GPT 5.4 produced no measurable speedups despite consistent correctness. The study separates two requirements that a coding score can obscure: preserving the answer and reducing the time needed to compute it. Study abstract.

Key takeaways · AI-distilled

Getting correct parallel code out of an agent took modest prompting. Getting code that actually ran faster was the hard part. Correctness and speedup are separate outcomes, and benchmarks usually measure only the first.
Across sorting, graph traversal, and search, one agent produced consistently correct parallel implementations with no measurable speedup at all, while another delivered the strongest gains. Accuracy-only scoring makes those two look identical.
No language won everywhere: C++ parallelized most reliably for graph algorithms, while Python and Julia produced the largest speedups on search. Python and Julia gained on some graph algorithms and regressed on others.
The paper's argument is that runtime performance belongs in coding benchmarks next to accuracy. For parallel work a passing test says nothing about whether parallelizing helped or made things slower.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Key quotes

“We find that coding agents can produce correct parallel implementations with modest prompting effort, but that achieving meaningful speedup is heavily algorithm- and language-dependent.”

“Sonnet 4.6 delivers the strongest overall performance gains, whereas GPT 5.4 produces no measurable speedups despite consistent correctness.”

“C++ is most consistently parallelizable for graph algorithms, while Python and Julia achieve the largest speedups on search algorithms: no single language dominates across all categories.”

“However, recent initial results suggest their parallel programming capabilities lag behind serial programming capabilities.”

Read the source arxiv.org