Vibeleaderboard
← All Intel
Intel / article

Measuring the sloppiness of code

Source
earendil.com
Author
doppp
Date

Context

Code can pass tests while becoming harder to extend. The essay on measuring code sloppiness explores ways to detect that deterioration, including the share of duplicated or flagged lines and the share of complexity concentrated in a few high-complexity functions. These measurements make aspects of maintainability visible that a functional pass rate can miss.

The underlying SlopCodeBench research evaluates agents repeatedly extending their own previous solutions as requirements change. That setup makes early design choices matter to later work. Its measurements track how code structure evolves across those changes, giving developers a way to investigate accumulating maintenance problems rather than judging each generated feature in isolation.

Terms in this piece · Glossary
  • LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Read the source earendil.com
Recommended reads
Comments

Checking sign-in…

Loading comments…