LLMs still produce bugs, but those bugs are different than what they used to be.
- Source
- x.com
- Date
LLMs still produce bugs, but those bugs are different than what they used to be. It’s less off-by-ones and more about system design, ui usability, missing broader context. Some kinds of coding has been solved, but not all. While models continue to improve, adversarial code review has been an incredibly powerful tool to catch many of these kinds of bugs. It can be as simple as a one line prompt - “use a dynamic workflow to adversarial test every edge case in an iOS simulator”, or use Claude’s built in /code-review (or /code-review low, /code-review medium, etc.)
Context
Boris Cherny says the bugs in AI-written code have changed kind. In his post, there are fewer off-by-one errors, the small slips where a loop or index runs one step too far or stops one step short, and more problems with system design, interface usability and missing broader . Those are mistakes in how the code fits the rest of the product rather than in a single line, and he sums it up by saying that some kinds of coding have been solved, but not all.
While models keep improving, his answer is adversarial code review: checking the work specifically for ways it breaks rather than to confirm that it runs. He calls it an incredibly powerful way to catch many of these bugs and says it can take a single prompt, giving the example “use a dynamic workflow to adversarial test every edge case in an iOS simulator”, or running Claude's built-in /code-review command, which also comes in variants such as /code-review low and /code-review medium. The post reports his own experience and offers no measurements of how bug types have shifted or how many such bugs the review catches.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Checking sign-in…
Loading comments…





