Tencent open-sources Hy-Nomas interactive video world model
Source
github.com
Author
Tencent Hy
Date
Why it matters
Tencent released weights, training code and Hy-Nomas-Bench for a model that generates video worlds segment by segment from text and camera input. It runs on one RTX 4090 with INT8 and offload, so teams can try and fine-tune it locally.
Key takeaways · AI-distilled
Hy-Nomas generates a video world one segment at a time, each segment steered by a free-form text prompt or a camera trajectory, and keeps a lifelong KV cacheThe memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.Full definition → across segments so the world stays consistent over many turns.
Tencent's single RTX 4090 mode stores DiT weights and the KV cache in INT8 with BF16 compute, bounds history with a sliding window, and moves the text encoder, DiT and VAE between CPU and GPU per stage, which needs ample system RAM and adds transfer overhead.
fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → trains LoRAA cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.Full definition → on the non-distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → base checkpoint (rank 256, Muon, 1e-4 for 2,000 steps in the shipped config), then merges the adapter into the distilled checkpoint used for fast inference. The documented example runs on 128 GPUs.
Besides the base and distilled models, the release includes a cinematic LoRA, a robotics LoRA trained on the RealSource-World robotic-arm dataset, and Nomas-PE, a model that turns user instructions into structured generation prompts for the demo.
Terms in this piece · Glossary
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
