The War Against Slop and an Emerging Weapon to Fight it: Formal Verification
- Source
- latent.space
- Author
- Richard MacManus
- Date

Context
Richard MacManus's Latent Space article presents formal verification, proving with mathematical logic that software does exactly what a precise specification says, as a remedy for low-quality AI-generated code. The distinction it draws comes from Clark Barrett, a Stanford computer scientist and co-creator of cvc5, an automated reasoning engine that Barrett says is now called billions of times a day in production pipelines at Amazon. When coding agents say they verified their work, they usually mean they ran tests, critiqued their own output or had another model judge it. Barrett's point is that those checks find many problems but never tell you when you are done, because another bug may be lurking that the tool cannot see. A proof settles the question for every property it states.
Several companies now pair AI with Lean, a language whose checker verifies each step of a proof, so the AI proposes the proof and the checker decides whether it holds. Barrett's company &Truth wants to read code in any language and prove whether it meets stated requirements, and Barrett names checking the world's C code for memory safety as one powerful use. Harmonic's Aristotle and Axiom Math's AxiomProver generate Lean proofs, and Theorem also focuses on security. The most concrete evidence on cost comes from a post by Anthropic's Frontier Red Team on work done in collaboration with Theorem: on 36 small verification tasks built from real open-source code, Claude Opus 5.5 proved half while using about 40% fewer than Opus 5. That is a trend on a narrow test, and the article still calls the approach token-expensive today.
For developers, the promised change is in where effort goes. In the workflow Barrett describes, which is a plan rather than a demonstrated product, a developer still asks an to build something but first works with it to pin down the specification; the agent then works with the verification tool behind the scenes, and the code should work the first time. Because a proof only guarantees what the specification says, the developer's time shifts toward writing and reviewing precise requirements instead of reading code. The article is a survey of company claims built around one interview, not an , so it shows where this field is heading rather than showing that these tools are ready for everyday projects.
- Clark Barrett, &Truth co-CEO and co-creator of the cvc5 SMT solver, says the gap between testing and formal verification is knowing when you are done: tests and scanners find many problems but cannot show that none remain.
- Lean works as a strict proof checker. Harmonic says that inside its Aristotle prover the AI writes proofs and Lean checks every step, so a result never rests on the AI being right.
- In a post with Theorem, Anthropic reported that on 36 small verification tasks Opus 5.5 reached a 50% solve rate with about 40% fewer tokens than Opus 5, a sign that token-expensive formal verification is becoming more token-efficient.
- Barrett names checking C code, open or closed source, for memory safety as a key &Truth use case, with the tool pointing to the specific code that must be fixed when it is not memory safe.
- &Truth pitches "veri-coding": the developer pins down a specification with the agent, the agent works with the verification tool, and more of the developer's time goes to writing and reviewing specs than to code.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Checking sign-in…
Loading comments…


