LoongForge

Why it's ranked here

Baidu's cloud training team open-sourced this from an internal suite it says has trained customer models on runs of over 5,000 accelerators, and the model list is unusually current: DeepSeek-V4, Kimi-K3, GLM-5.2, Qwen3.8 and nine robot-control model families. It is Apache 2.0 licensed, with test suites for both engines in the tree. The headline speed numbers, 4.38x throughput on one robot model and up to 74% lower overhead from its expert load balancer, are the project's own measurements, backed by its own paper but not independently reproduced. Packaging is rougher than the training code: the package declares itself beta and its command-line entry does not resolve.

What's good

The two-engine split is the design call worth noticing. Large language and multimodal models want Megatron's tensor and pipeline splitting, while robot models are served by plain PyTorch data-parallel and fully sharded training, and neither is forced onto the other. Vision-language models can be assembled in config from a separate vision encoder and language model, with each half given its own parallelism and freezing settings. Dependencies are pinned to exact versions per hardware target. Checkpoint conversion to and from Hugging Face is a first-class tool with per-model weight mappings, which matters when you need the trained weights somewhere else afterward.

Tradeoffs

Nothing installs by default: the base package declares no dependencies, so you pick the GPU, Kunlun or robot install set, and those sets disagree. They pin three different Hugging Face Transformers versions, so one environment cannot serve both the language and robot stacks. The robot set also pins tooling that reads like a frozen developer machine, including a Chinese JupyterLab language pack and pre-commit. The Megatron engine depends on a Baidu fork, so upstream Megatron changes reach you only through that fork. The packaged command points at a function the training module does not define, and that module's export list has a missing comma that breaks a wildcard import.

How to use it well

Fits a team already running multi-node training on NVIDIA or Kunlun hardware that wants tuned recipes for current open-weight models, especially mixture-of-experts models, where its expert balancing and communication overlap apply. Start from the shipped config and launch script for your model rather than the declared command, and use the project's Docker image, which carries every model family in one image, instead of resolving the pins by hand. Benchmark the speed claims on your own cluster before budgeting around them. It is aimed at clusters rather than one-card adapter fine-tunes, and it does not serve models for inference.

Observed

License

Apache-2.0 (LICENSE file and package metadata)

Language

Python, requires 3.10 or later

Packaging

hatchling build; base package declares no dependencies, installs via gpu, xpu or vla extras

Hardware targets

NVIDIA GPUs and Baidu Kunlun XPUs, with separate pinned dependency sets

Training backends

Patched Megatron-LM fork for LLM, VLM and diffusion; torch-native DDP/FSDP trainer for embodied models

Declared maturity

Development Status 4 - Beta classifier

Command-line entry

Declared as loongforge.train:main; the train package read defines no main

Checkpoint tooling

Hugging Face to Megatron checkpoint converter shipped under tools

Tests

Test directories for the embodied and LLM/VLM stacks in the tree; pytest in the test dependency group

Distribution

Prebuilt Docker images on Docker Hub, one image across model families