Do Dynamic Languages Really Save Coding Agent Tokens?

Dan Luu set out to test a claim that keeps showing up in search summaries: that dynamically typed, concise languages cost fewer tokens when an AI agent writes the code. Luu's essay finds that claim does not survive tasks larger than toy problems. The post behind it measured Rosetta Code exercises that some languages solve in about 70 to 110 tokens, and Luu argues results on tasks that small do not carry over to real work. Luu also found a scoring bug in a second comparison that blamed Rust's failures on the language: one test called a file path that did not exist, and when Luu rescored Rust against its own program, it got a perfect score.

Luu then ran two evals with GPT-5.6 Sol in the Codex agent, at medium and ultra reasoning effort, plotting cost against correctness. In the first, agents with no internet access implemented a decoder for Zstd, a compression format, from its written specification, and were scored on tests they never saw. At medium effort the dynamic languages clustered ahead, which looks like support for the claim. At ultra effort the picture was mixed, with a couple of static languages doing best. The second task had agents reimplement Pandoc, a document-format converter, with a provided test suite, then scored them on a separate set of held-back tests, and again showed no strong split between static and dynamic languages. Across both, obscure languages and assembly tended to do poorly, and more popular languages showed a weak to moderate tendency to produce correct code more cheaply. That runs against the earlier suggestion that dense languages like J are the efficient choice.

Small, idiosyncratic failures drive a lot of these scores, which is why Luu refuses to rank individual languages. In the Zstd runs, 36 of 40 medium-effort and 5 of 40 ultra-effort Clojure programs failed tests because of one byte-conversion mistake. Luu counts that as a real result, since tests can catch such a bug but fixing it still burns time and tokens. Luu also notes that the tests do not measure memory safety; a quick check of the Pandoc programs found memory-safety problems in every C program and all but one C++ program.

Luu's own caveats are heavy. Agents set up the evals, Luu says they fixed more than 100 setup bugs and expects more remain, and two tasks can refute a sweeping rule without establishing which language is best. In an update Luu points to a paper by Tom Adamczewski and coauthors from Epoch AI and others that tests many more tasks but fewer languages and concludes language does not matter; Luu thinks testing fewer languages explains that conclusion and suggests reading the two together. The practical advice Luu draws is narrow: for an ordinary user of these tools, a mainstream language is likely a better bet than an obscure dense one.