Awex syncs trillion-parameter weights to inference in under six seconds
- Source
- x.com
- Date
💥New Release: Awex, a high-performance #RL training-inference weight synchronization framework. #opensource 🚀 It cracks the tough nut of training/inference weight params latency, syncing 1T-param models across thousands of GPUs in less than 6 seconds!
Join Awex https://t.co/gwotBgBIk6 and check out these amazing features!⬇️ ⚡ Blazing Sync: Trillion-param models synced across 1000-GPU clusters in 6s, top-tier performance. 🔄 Unified Adapt: Auto-manage parallelism & tensor format/layout diffs, multi-model compatible. 💾 Zero-Redundancy: Transfers only needed shards, in-place updates, no realloc & copy hassle. 🚀 Multi-Mode: NCCL, RDMA, shared memory modes, max out NVLink/NVSwitch/RDMA, cut latency. 🔌 Heterogeneous: Works in co-located/separated, sync/async RL, RDMA for dynamic scaling. 🧩 Pluggable: Custom weight sharing, layout, integrate new train/inference engines.
Context
In large reinforcement-learning training runs, the trainer periodically has to hand a fresh copy of the model's weights to the separate machines running , and at trillion-parameter scale across thousands of GPUs that hand-off can itself become the bottleneck. Ant Open Source says its Awex framework syncs a trillion-parameter model's weights across a thousand-GPU cluster in under six seconds, transferring only the shards that actually changed and updating them in place instead of reallocating memory. The team reports it works across co-located and physically separated setups, and in both synchronous and asynchronous RL training.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Checking sign-in…
Loading comments…