QwenGyre: Elastic RL Infrastructure for Hours-Long Agent Rollouts

Qwen's QwenGyre paper tackles a scheduling problem that appears when reinforcement learning is applied to agents whose tasks take hours (paper). Training alternates between rollouts and updates. In a rollout, the model works through tasks inside an agent harness. In an update, it learns from the scored results. The paper's main workload is NL2RepoBench, which asks an agent to build a software repository from a written specification. There, the smaller Qwen 3.6 122B model averaged 1.93 hours per rollout. For the flagship Qwen 3.8 2.4T, 61.1% of queries with complete timing records took four hours or more. The two usual setups waste GPUs on work like this. Colocate switches one shared pool between rollout and training as a unit, so training waits for the slowest task even while GPUs sit idle. Async gives each job its own fixed pool, so training GPUs wait for a batch while rollout cannot borrow them.

QwenGyre splits the GPUs into cells that can each switch roles. As tasks finish and the remaining rollout work fits on fewer cells, cells move to training at different points and join the batch already in progress. Meanwhile, the remaining agents' model requests are rerouted to other cells. Each agent's workspace and tool state stay where they are. That matters because the harness interface, with Claude Code in these experiments, offers no general way to checkpoint and resume a live task. On the flagship run, 48 training steps took 75.42 hours with QwenGyre, against 134.55 hours for Async and 91.47 hours for Colocate, with equal GPU budgets and matching training scores. The announced saving of roughly 44% is the comparison with Async. The speedup over Colocate was 1.21 times. The authors link that smaller margin to task durations bunching near the timeout limit, which leaves fewer chances to free up cells early.

The second component deals with what long agent runs produce. Harnesses compact full context windows, hand subtasks to sub-agents and retry failed paths, so one task yields a branching set of conversations rather than a single transcript. QwenGyre stores these as a tree whose branches share their beginnings. It trains only on tokens the model itself generated, counts shared ones once, and keeps at most five paths per task, prioritising the main agent. In the ablation, the cap of five matched the training scores of using every path while cutting forward-backward computation time by 25.2%. Tasks whose harness hits its time limit are still scored on whatever work they left in the workspace, instead of all counting as identical failures.

The announcement says QwenGyre lifts NL2RepoBench from 52.5% to 58.5%. The paper reports that rise, from 52.48% to 58.54% over the 48 steps, as the result of the flagship training run, and it does not show it as an advantage over the baselines, whose training scores QwenGyre matched. What the paper demonstrates is faster training at the same quality. It covers three workloads run through Claude Code: NL2RepoBench, DeepSWE and TerminalBench. Across them, it reports speedups of 1.38 to 1.78 times over Async and 1.21 to 1.85 times over Colocate. The authors note that much smaller GPU budgets were not evaluated.