Towards Effective Process Supervision in Mathematical Reasoning

A correct mathematical answer can conceal an invalid derivation. Qwen's ProcessBench tests whether an AI evaluator can find the first faulty step, rather than merely recognize a correct final answer. It contains 3,400 worked solutions with error locations annotated by experts, concentrated on competition and Olympiad mathematics. An evaluator must identify the earliest error or judge the entire solution correct. The release also provides two models trained to score intermediate reasoning steps.

This addresses a weakness in testing evaluators only by whether they select a successful answer from several attempts. That selection can succeed even when the evaluator misses flawed reasoning. ProcessBench makes error detection itself the task, providing a more direct test of whether step-by-step feedback deserves trust.