LongLive
github.com/nvlabs/longlive- Category
- AI Tools
- Rank
- No. 871Tools index
Previous survey · No. 853 ·
- Type
- APP
- Builder
- nvlabs
- GitHub
- 2.6k stars
- Date
About
NVIDIA LongLive — infrastructure for long-form AI video generation.
What it does
LongLive turns Wan2.2 video models into autoregressive text-to-video and image-to-video systems. It supplies training, distillation, quantized inference, sequence parallel execution, asynchronous decoding, and multi-shot prompt handling. Users configure checkpoints and data in YAML, then run distributed commands or assemble the inference pipeline from Python.
Why it's ranked here
This is a substantial research implementation, not a thin model wrapper. It covers teacher-forced training, DMD distillation, BF16 inference, two low-precision paths, and distributed execution. The narrow model target and demanding NVIDIA setup reduce general usefulness, but teams working on efficient autoregressive video get unusually complete machinery.
What's good
The strongest feature is continuity across the full model workflow. Text-to-video and image-to-video share a two-stage training recipe, while inference supports attention sinks, cached state, asynchronous decoding, and multi-shot prompts. Quantization is treated carefully: sensitive projections remain in BF16, incompatible modes are rejected, and checkpoint handling differs explicitly between packed FourOverSix weights and TransformerEngine runtime quantization.
Tradeoffs
The release accepts only the Wan2.2 TI2V 5B model family. It assumes CUDA hardware, while validated FP8 use requires compute capability 8.9 or newer. NVFP4 needs a separate, version-sensitive environment plus locally built CUDA extensions. Sequence-parallel inference omits image-to-video and FP8, and disables quantized key-value caches. Compilation warm-up can take minutes, with longer generations introducing new cache shapes and possible eager fallback.
How to use it well
Use LongLive when a research or infrastructure team needs to train, distill, quantize, and benchmark one supported autoregressive video stack on NVIDIA GPUs. Start with BF16 for smoke tests, then adopt FP8 or NVFP4 for repeated inference. It is not a general video application, broad model-serving layer, or retrieval system. Retrieval augmentation lives in a separate project.
Technical notes+
train.py selects ScoreDistillationTrainer or DiffusionTrainer from normalized OmegaConf configuration and defaults to automatic checkpoint resume. inference.py provides distributed NCCL execution, BF16, TorchAO FP8, NVFP4 checkpoint handling, optional torch.compile, image-to-video datasets, and video output. inference_sp.py implements Ulysses sequence parallel groups but explicitly rejects FP8 and I2V while disabling KV quantization. model/base.py restricts construction to Wan2.2-TI2V-5B, and model/dmd.py implements distribution matching with blockwise timesteps and I2V context masking. utils/fp8.py applies row-wise dynamic W8A8 to eligible BF16 linear layers and preserves selected projections. utils/quant.py contains FourOverSix and TransformerEngine NVFP4 paths. docs/getting_started.md documents separate BF16 and NVFP4 environments; requirements.txt exposes the pip dependency surface.
Observed
- License
- Provided source files declare SPDX-License-Identifier: Apache-2.0.
- Primary language
- Python, with YAML-driven configuration and local CUDA extensions for NVFP4.
- Install surface
- Conda environment plus pip requirements; FlashAttention is installed separately.
- Interfaces
- Command-line training and inference entry points, plus an importable Python inference pipeline.
- Platform support
- CUDA-based NVIDIA GPU execution; Flash Attention 3 targets Hopper and falls back to Flash Attention 2 elsewhere.
- Model scope
- The provided model construction code accepts only Wan2.2-TI2V-5B.
Read from README.md, requirements.txt, docs/getting_started.md, docs/FLASH_ATTENTION_3_AND_HOPPER_SUPPORT.md, train.py, inference.py, inference_sp.py, model/dmd.py, utils/fp8.py, model/base.py, utils/loss.py, utils/misc.py, utils/quant.py.
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.