Vibeleaderboard
← All Intel
Intel / article

Humanize: Judgement Engineering for Agentic Coding

Source
arxiv.org
Author
Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
Date

Context

Humanize is a coding-agent workflow built on the idea that the agent writing code is a weak judge of whether it is finished. According to the paper, a human approves a plan, one builder agent implements it in rounds, and a reviewer agent from another vendor decides when the work is complete. Fixed hooks, not a , route work between roles and enforce 72 mechanical gates.

The authors argue that alternating two models means a defect survives only if both miss it. They study the system through 118 public postmortems of real runs, and report that catches unsupported builder claims while stopping remains a weakness: in reports that separate rounds by phase, two thirds of rounds happened after implementation had been accepted.

The authors say this evidence is observational, not a controlled comparison of workflows.

Key takeaways · AI-distilled
  • In Humanize, a human approves the plan contract, a builder agent implements it in rounds, and deterministic hooks rather than a model route work between roles and enforce 72 mechanical gates.
  • The postmortems show independent review catches unsupported builder claims, but stopping remains weak: in reports that split rounds by phase, two thirds of rounds came after implementation was accepted.
  • The authors report full scores on IOI, IMO and IPhO 2026 through Olympiad Agents, 672/672 on PutnamBench, and top-three finishes in all three Full-Agent tracks of the MLSys 2026 FlashInfer contest.
  • The authors caution that their evidence is observational, not a controlled comparison of workflows; the went through 68 versions in 108 days.
Terms in this piece · Glossary
  • Model — A computational system that uses learned patterns to produce an output from information supplied to it.
  • Independent review — Assessment of a change by a person, a check or a reader that did not produce it.
  • File — A named unit of stored data that keeps its contents while no program is using it.
  • Tool — An operation that the software around a model makes available for the model to request, with a name, expected inputs and a kind of result.
Recommended reads
Comments

Checking sign-in…

Loading comments…