Intel / article

MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis

Source: arxiv.org
Author: Ximing Lu, Aviral Kumar, Jan Kautz, Yejin Choi, David Acuna
Date: Published Oct 12, 2026

Context

MagicBeanStalk addresses a training problem: when every sampled answer to a question gets the same pass-or-fail reward, a method that compares those rewards has no difference to learn from. Adding more questions of that kind need not improve training.

The algorithm breaks difficult problems into simpler components, generates new variants and recombines them. It keeps problems that produce both successful and failed attempts, including a complete correct solution within the allowed response length. The authors report that training OpenAI's gpt-oss-120b on this data raised its average accuracy across eight math benchmarks from 61.2% to 69.8%, evaluated with a 16,000- output limit. The result supports matching training difficulty to the model and its response budget.

Key takeaways · AI-distilled

The authors argue filtering out zero-advantage problems can only select, so it discards the hard problems; useful synthetic data should exercise the reasoning those hard problems need while staying so some rollouts succeed and some fail.
They report that simply prompting a teacher model to vary hard problems does not produce well-calibrated difficulty; MagicBeanStalk instead decomposes seed problems top-down and regrows them bottom-up while keeping each seed's reasoning blueprint.
On 8 math benchmarks at a 16K-, continual RLVR on MagicBeanStalk data lifted gpt-oss-120b from 61.2 to 69.8 and DeepSeek v4Flash from 72.4 to 77.9; training on the full 78K source corpus gained only +1.1.
In finance, the authors synthesized over 14K verifiable problems from 340 solutions and report gains of 6.9 points on SecQue and 2.4 on FinanceBench over the base model.

Terms in this piece · Glossary

token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
token budget — A cap on how many tokens a task, session, or agent run may consume — the practical control on both cost and how long an agent will grind.

Read the source arxiv.org