Introducing AState, the 3rd #OpenSource component to ASystem!
AState is a general state data management system for #reinforcementlearning tasks. It is currently deployed in Ling-1T RL training tasks, achieving a ⚡️breakthrough in cutting trillion-parameter weight synchronization time from minutes to ~6 seconds🚀
More features addressing critical issues like RL IO inefficiency and multi-session state recovery will be continuously open-sourced in subsequent releases. 👀Stay tuned!
Welcome to the ASystem's community!
📦 GitHub Repository: https://t.co/Qn9jd6on1v
For the awesome Ling/Ring models trained by ASystem:
🤗 Hugging Face: https://t.co/wJfOOugBGE
🤖 ModelScope Community: https://t.co/H1XGr5PkRW
Core Features 1/3
🔁Efficient Transfer: AState supports automatic weight mapping (resharding) between nodes(train and rollout actors), utilizing zero-redundancy and in-place updates.
This core mechanism significantly boosts trillion-parameter weight transfer efficiency, serving as the foundation for seconds-level synchronization.
Core Features 2/3
⚖️Architectural Stability: AState fully leverages NUMA topology and hardware affinity to reduce cross-device latency. Through global execution planning, it achieves more balanced load distribution, effectively mitigating traffic hot spots and long-tail latency in thousand-card RL tasks.
Weight synchronization between training and rollout actors is a main stall in large RL loops, and AState reports cutting it to roughly 6 seconds at trillion-parameter scale using resharding with in-place updates rather than full transfers.
