Large language model agents are increasingly being asked to perform extreme-long, or "xlong," horizon tasks, where a single execution can run for hours, involve hundreds of model-environment interactions, and consume nearly one million tokens per rollout. According to a newly posted research paper, applying online reinforcement learning (RL) to these long executions creates two fundamental obstacles.
The first is severe execution variance and prolonged rollout delays, which cause substantial GPU idling. The second is complex non-linear branching, which generates large amounts of redundant trajectory data and undermines training efficiency.
To address both problems, the paper introduces QwenGyre, described as an end-to-end framework for xlong-horizon online RL. The framework elastically reallocates GPUs between rollout and training without interrupting live executions. Its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths in order to bound training costs.
When scaled to the authors' flagship model, Qwen 3.8 2.4T, with 700K tokens per rollout, QwenGyre produced a 6.0% absolute gain on NL2RepoBench, rising from 52.5% to 58.5% in 48 steps, according to the paper.
Across evaluations on training datasets spanning diverse domains, the framework delivered speedups of up to 1.85× over a Colocate approach and 1.78× over an Async approach, the paper states.