- Category
- Developer Tools
- Rank
- No. 1074Tools index
Previous survey · No. 1053 ·
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run
- Builder
- @whitecircle
- GitHub
- 850 stars
- Latest release
- v1.2.0
- Date
About
Halo is an open-source framework built by White Circle for post-training large language and multimodal models. It keeps models in their native HuggingFace format while claiming up to 2.8x the throughput of the standard TRL library with lower peak memory usage.
What it does
Halo is a training stack for people who want to fine-tune or reinforce open-weight models on their own NVIDIA GPUs without rewriting those models for distributed hardware. You pick a method (supervised fine-tuning, preference tuning, distillation or reinforcement learning), point it at a YAML config, and one command chooses the right launcher for one GPU or eight. It splits mixture-of-experts layers, attention and weights across GPUs inside the original Hugging Face model, so the saved checkpoint opens in standard tooling afterwards. Reinforcement learning runs rollouts on separate vLLM or SGLang servers and pushes fresh weights to them during training.
Why it's ranked here
The case rests on engineering depth that is visible in the repository, not on the headline numbers. The build pins exact NCCL, Flash Attention and DeepEP versions per GPU generation and explains each pin, the test suite is split into CPU and GPU tiers with a manifest launcher, and the launcher refuses bad input with specific errors. The throughput claims (2.6 to 3.5 times stock TRL on gpt-oss-20b, eight B300s) are the vendor's own benchmarks against a baseline it configured. The custom license and the narrow hardware target keep it out of reach for many teams, so it suits serious GPU owners more than casual tinkerers.
What's good
Models stay ordinary Hugging Face models: parallelism wraps the existing layers instead of replacing them, so there is no conversion step at either end. The launcher turns a method name into the right python, torchrun or accelerate command, passes extra flags through as config overrides, offers a dry run that prints the command, and rejects multi-node torchrun flags with a message pointing at the multi-node guide instead of letting them fail inside the trainer. New scripts register themselves from the folder layout. Prebuilt Hopper and Blackwell images pull without an account, and the docs come in a short human guide plus a full reference written for coding agents.
Tradeoffs
The license is Apache 2.0 with supplemental terms. Offering Halo as a hosted training service past 20 million dollars in yearly revenue requires a commercial agreement with White Circle, and companies above that revenue must credit Halo on released models. Hardware is NVIDIA only, built for Hopper and Blackwell, and Python is pinned to 3.12. The dependency list is long and tightly capped, including wandb, ClearML, Gradio and boto3, so installing outside the Docker image is not the intended path. The training image grants passwordless sudo to every user for SkyPilot. The launcher is single-node; multi-node jobs need one torchrun per node by hand.
How to use it well
Use it if you own or rent H100, H200, B200 or B300 machines and want to post-train a mixture-of-experts or long-context model while keeping standard checkpoints. Start from a shipped example config, run it once with the dry-run flag to see the exact command, then override fields on the command line. For reinforcement learning, run the rollout server on GPUs the trainer does not use. Read the license before building a hosted fine-tuning product on it. It does not cover serving, evaluation dashboards or consumer GPUs, and it is heavy for a quick LoRA run on a laptop.
Technical notes+
pyproject.toml declares license LicenseRef-Halo with license-files pointing at LICENSE and a bundled Apache text, requires-python >=3.12,<3.13, a single console script halo = src.cli:main, and narrowly capped pins on torch 2.11, transformers 5.16 and trl 1.6; uv override-dependencies pins nvidia-nccl-cu13==2.31.2 exactly so the training, vLLM and SGLang images agree on the weight-sync communicator. src/cli.py is a Typer app: training_methods() and tool_commands() index scripts under scripts/training and the other scripts subtrees by walking the tree and keeping files with a __main__ guard; command() selects accelerate launch, torchrun --nproc_per_node or sys.executable; reject_multinode_flags() refuses --nnodes, --rdzv-* and friends. Dockerfile builds FA4 for Blackwell or FA2+FA3 from source for Hopper (stubbing 24 split-K kernels that hang CUDA 13.2 ptxas), builds DeepEP at a pinned commit, sets TORCH_ALLOW_TF32_CUBLAS_OVERRIDE=0 because TF32 corrupts long-context RoPE, and appends ALL ALL=(ALL) NOPASSWD:ALL to sudoers. Makefile exposes test-cpu, test-gpu-core, test-gpu-vllm, test-gpu-sglang and bench targets, all run inside the image. human-docs/cli.md documents halo launch and halo run.
Observed
- License
- Apache 2.0 with Halo Supplemental Terms (hosted training service threshold and attribution clause)
- Language
- Python, pinned to 3.12
- Install surface
- Prebuilt Docker images per GPU generation (Hopper, Blackwell), or local build via make
- Interface
- halo CLI with launch and run verbs; training scripts also runnable directly
- Hardware
- NVIDIA GPUs; images built for H100/H200 and B200/B300
- Training methods
- Pre-training, SFT, preference optimization, distillation, online and async multi-turn RL
- Rollout engines
- vLLM or SGLang in separate containers
- Multi-node
- Launcher is single-node; multi-node runs use one torchrun per node
- Tests
- CPU tier and GPU tiers run inside the image via Makefile targets
Read from README.md, pyproject.toml, human-docs/cli.md, src/cli.py, LICENSE, Makefile, Dockerfile.
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.