Humanize: Judgement Engineering for Agentic Coding
- Source
- arxiv.org
- Author
- Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
- Date

Context
Humanize is a coding-agent workflow built on the idea that the agent writing code is a weak judge of whether it is finished. According to the paper, a human approves a plan, one builder agent implements it in rounds, and a reviewer agent from another vendor decides when the work is complete. Fixed hooks, not a , route work between roles and enforce 72 mechanical gates.
The authors argue that alternating two models means a defect survives only if both miss it. They study the system through 118 public postmortems of real runs, and report that catches unsupported builder claims while stopping remains a weakness: in reports that separate rounds by phase, two thirds of rounds happened after implementation had been accepted.
The authors say this evidence is observational, not a controlled comparison of workflows.
- In Humanize, a human approves the plan contract, a builder agent implements it in rounds, and deterministic hooks rather than a model route work between roles and enforce 72 mechanical gates.
- The postmortems show independent review catches unsupported builder claims, but stopping remains weak: in reports that split rounds by phase, two thirds of rounds came after implementation was accepted.
- The authors report full scores on IOI, IMO and IPhO 2026 through Olympiad Agents, 672/672 on PutnamBench, and top-three finishes in all three Full-Agent tracks of the MLSys 2026 FlashInfer contest.
- The authors caution that their evidence is observational, not a controlled comparison of workflows; the went through 68 versions in 108 days.
- Model — A computational system that uses learned patterns to produce an output from information supplied to it.
- Independent review — Assessment of a change by a person, a check or a reader that did not produce it.
- File — A named unit of stored data that keeps its contents while no program is using it.
- Tool — An operation that the software around a model makes available for the model to request, with a name, expected inputs and a kind of result.
articleNot All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the WildObada Kraishan
articleZero2Repo: Can Coding Agents Build Repositories from Scratch?Pei Yang, Tianyu Shi, Yuhang Yao, Wanyi Chen, Tongyun Yang, Dun Pei, Haonan Wang, Pengbin Feng, Guanxu Yu, Jingchun Huang, Zeyu Zhang, Shuhan Sun, Hao Li, Xiang Li, Jie Xiao, Xinyu Wang, Hanxin Chen, Daqi Li, Qi Jia, Hongshan Lin, Zhizhou Gu, Zijun Tian, Weizhi Du, Lynn Ai, Eric Yang
articleSWE-Duel: Towards a Self-Scaling Arena for Coding AgentsWei Minn, Zhensu Sun, Lwin Khin Shar, David Lo
Checking sign-in…
Loading comments…