AstraFlow is a dataflow-oriented reinforcement learning system designed for better flexibility and scalability.

AstraFlow natively supports the following for LLM RL training without any feature-specific system engineering:

• Fully Async Multi-policy collaborative RL
• Elastic heterogeneous cross-region rollouts
• Substitutable rollout and trainer service
• Composable data algorithms

Fully Async Multi-policy Collaborative RL Training: multiple policies train together, each as an independent trainer with its own data and weight stream.

News

• [2026/07] AstraFlow v0.1.2 images released — transformers 5.8 / SGLang 0.5.13 stack, new Qwen3.5-4B (Gated-DeltaNet) math recipes, Megatron-image build fix. Both images validated end-to-end on 8×H100 (400-step math-RL runs incl. eval). See docker/README.md.
• [2026/06] New recipe: dynamic recursive agent on TextCraft — a multi-turn agent that recursively spawns sub-agents sharing inventory under a team reward. See the recipe docs.
• [2026/06] AstraFlow v0.1.1 released — CUDA 13 image, SGLang 0.5.12, Megatron weight-sync training backend, and transformers 5 support. See the project website.
• [2026/05] AstraFlow v0.1.0 released — first public release of the full system. See the project website.
• [2026/05] AstraFlow paper is on arXiv.

Getting Started

• Install AstraFlow
• Quick Start

Recipes

| Recipe | Description | |---|---| | math/ | RLVR math reasoning — Qwen3-1.7B / 8B, M2PO, full and delta-weight transfer | | math-multi-agent/ | Actor + verifier collaborative math training | | math-efficient-data/ | Composable data algorithms — GRESO, dynamic sampling, buffer replay | | code/ | Code-generation RL — Qwen3-8B, M2PO | | code-multi-agent/ | Codegen + verifier competitive coding | | search/ | Search-augmented agent training with local retrieval | | alfworld/ | ALFWorld embodied household agent | | webshop/ | WebShop web-navigation shopping agent |

Roadmap

• [ ] Offline cluster training — Support training on offline clusters without internet access.
• [ ] All-in-one launcher — A launcher helper that streamlines bringing up the AstraFlow, RaaS, and trainer services.
• [ ] MoE model support — Extend the training backends to Mixture-of-Experts models.
• [ ] Terminal-Bench training — Add a recipe for training agents on Terminal-Bench.
• [ ] Megatron backend — Add Megatron-LM as a training backend.
• [ ] vLLM rollout engine — Support vLLM alongside SGLang as a rollout engine.

Citation

@article{zheng2026astraflow,
  title   = {AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs},
  author  = {Zheng, Haizhong and Di, Yizhuo and Wang, Jiahui and Jin, Shuowei and
             Liu, Xueshen and Wu, Yongji and Mao, Z. Morley and Stoica, Ion and
             Zhao, Jiawei and Chen, Beidi},
  journal = {arXiv preprint arXiv:2605.15565},
  year    = {2026}
}

Acknowledgment