← All IntelIntel / post
LLMs still produce bugs, but those bugs are different than what they used to be.
- Source
- bcherny
- Author
- bcherny
- Published
Terms in this piece · Glossary
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
It names the failure class that survives better models, which is design and rather than syntax. The adversarial review pattern is cheap to adopt and directly changes how you gate -written code.
Transcript
LLMs still produce bugs, but those bugs are different than what they used to be. It’s less off-by-ones and more about system design, ui usability, missing broader context. Some kinds of coding has been solved, but not all.
While models continue to improve, adversarial code review has been an incredibly powerful tool to catch many of these kinds of bugs.
It can be as simple as a one line prompt - “use a dynamic workflow to adversarial test every edge case in an iOS simulator”, or use Claude’s built in /code-review (or /code-review low, /code-review medium, etc.)
Read the source ↗x.com
More from bcherny
Recommended reads
- clipLLMs are terrible at debugging — so founders reroll insteadY Combinator
articleTo Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code EditingAmir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia, Gopi Krishnan Rajbahadur, Ahmed E. Hassan
articleAdversarial Attacks on LLMsLilian Weng
Comments
Sign in to comment.
Loading comments…