Zero RL at trillion scale, with no format rewards required

'Zero RL' refers to training a reasoning model directly from a pretrained base using reinforcement learning, without a supervised fine-tuning stage on human-written examples first. Ant Ling introduces Ring-2.5-1T-Zero, which the company describes as scaling this Zero RL approach to a trillion-parameter model for the first time. Ant Ling reports that on smaller models, complex reward shaping is needed to push a model toward structured reasoning, but that at 1 trillion parameters the model developed reasoning behaviors on its own with a simpler reward setup, including what the company characterizes as spontaneous parallel exploration of alternate solution paths.

The company describes a four-stage training pipeline: growing chain-of-thought length with token-level reinforcement learning, distilling that into a more concise model with supervised fine-tuning, refining further with sample-level reinforcement learning, and finally routing between reasoning depths adaptively. Ant Ling reports the resulting model solves AIME problems using less than half the token budget of baseline models it compares against, and that distilling 100,000 of its own reasoning traces into a much smaller Qwen-32B model outperformed distillation using 800,000 traces from DeepSeek-R1. These are the company's own reported comparisons; the arXiv paper's full experimental setup was not independently reviewed for this explanation.