QwenGyre Operational

QwenGyre is Alibaba Qwen's online reinforcement learning framework for ultra-long-horizon agents, designed for tasks of approximately 700,000 tokens per run lasting several hours. QwenGyre uses elastic scheduling to switch idle GPUs between sampling and training without interrupting tasks, while trajectory processing preserves real context, evaluates partial results, and deduplicates shared prefixes to control costs. On Qwen3.8-2.4T, it raised the NL2RepoBench pass rate from 52.48% to 58.54% in 48 steps, achieving 1.78× end-to-end speedup over Async.

Ultra-long-horizon online reinforcement learning: Directly trains with RL on tasks lasting several hours per run and totaling approximately one million tokens.
Elastic GPU scheduling: Dynamically switches idle GPUs between sampling and training without interrupting tasks.
Request rerouting and KV cache migration: Agents continue running when GPU roles switch, with workspaces and tool states preserved; KV caches are migrated through RDMA to reduce duplicate computation.
Streaming training and mid-run node joining: Core cells maintain authoritative parameters, while satellite cells can pull snapshots at any time to join batches in progress and claim micro-batches according to progress.
Trajectory trees and real-context preservation: Token-in Token-out records the exact input and output of every call, organizing them into trajectory trees by shared prefixes so outputs always correspond to the context in which they were generated.
Partial Scoring: Tasks that time out before completion can still be scored under task-specific rules when they produce verifiable results, and the scores can be used for training.
Trajectory budgets and loss balancing: Each execution retains at most Jmax trajectories, and shared prefixes count toward the loss only once, preventing tasks with many branches from receiving excessive weight.

Elastic Scheduler: GPUs are divided into several homogeneous compute units. Each cell can provide sampling services through SGLang or host a complete training replica with Megatron. The scheduler tracks the “watermark,” or the amount of unfinished work. As long as the remaining sampling capacity can support running tasks and the supply of training data is sufficient, it switches cells into training one by one in order. During switching, an agent reroutes model requests to the new inference engine and retries unfinished requests; backend processes and communication groups do not need to be rebuilt. Core cells hold authoritative weights and optimizer states, while satellite cells pull parameter snapshots over RDMA and join gradient accumulation for the current batch. The core then aggregates and updates them, allowing training to scale gradually as data becomes ready while highly overlapping sampling and training.
Trajectory Processor: QwenGyre uses an agent to record the exact input/output token IDs and behavior-policy log probabilities for every model call. Shared prefixes are reused and branches are created at divergence points to form trajectory trees, ensuring every output always corresponds to its original generation context. During training admission, no more than Jmax trajectories are selected for each execution according to role priority. Only tokens generated by the policy model participate in the loss, and shared targets are counted once. The loss is first averaged over selected tokens within each execution and then averaged across executions, decoupling execution weights from the number of branches. Combined with Partial Scoring, this evaluates and scores completed portions of timed-out tasks, maximizing the use of sparse and costly valid signals from ultra-long-horizon executions.

QwenGyre is currently published in paper form; Alibaba has not yet released its code or installation package.

First online RL implementation for ultra-long-horizon tasks on a flagship model: It performs online reinforcement learning on approximately 700,000-token tasks per run with Qwen3.8-2.4T, raising the NL2RepoBench pass rate from 52.48% to 58.54% in 48 steps.
Major training efficiency gains: With the same GPU budget, it achieves up to 1.78× end-to-end speedup over Async and 1.85× over Colocate, reducing 48-step training time from 134.55 hours to 75.42 hours.
Elastic scheduling with zero task interruption: When GPUs switch dynamically between sampling and training, agent execution, workspaces, and tool states remain fully preserved. Switching takes only seconds, negligible compared with executions lasting several hours.
Maximized GPU utilization: Fine-grained cell-level resource allocation eliminates the long-tail idle waiting of Colocate and the rigid fixed partitioning of Async, allowing training to be fully hidden behind sampling.
Training with faithful real context: The TITO mechanism ensures every output corresponds to its original generation context, avoiding “false-context” training signals caused by historical compression and sub-agent context concatenation.

arXiv technical paper: https://arxiv.org/pdf/2609.33848

Natural-language generation of complete code repositories: Builds multi-package software repositories from scratch based on natural-language specifications, with approximately 700,000 tokens and runtimes of up to several hours per execution.
Long-horizon software engineering on existing repositories: Handles complex tasks such as feature development, bug fixes, and cross-file refactoring in existing codebases.
Multi-turn terminal automation: Agents continuously execute hundreds of tool calls in command-line environments to complete system configuration and operations tasks.
Large-scale codebase refactoring and migration: Handles engineering tasks such as cross-repository technology-stack upgrades and comprehensive refactoring that require hours of continuous planning and execution.
Multi-agent collaboration: Supports hierarchical execution in which a primary agent delegates to sub-agents; trajectory processing leverages branching structures to control the training cost of subtasks.