NVIDIA Nemotron Labs Dory1.0 4B BF16 released

Dory1.0-4B is a small NVIDIA language model built so its compute per token can be turned up or down after training. According to the model card, it is a looped, or latent-recurrent, model: an 8-layer input block and an 8-layer output block surround a shared 20-layer block that can run several times in a row, with the original input fed back in on each pass. More passes mean more computation on each token without adding parameters. The idea of reusing the same layers to think longer is covered in more depth in Sebastian Raschka's explainer on looped transformers.

The practical difference is that one checkpoint serves several budgets. Users set the loop count at serving time, from one to four in the llama.cpp instructions, with no retraining or model swap. NVIDIA recommends two loops as the default and says most of the gain arrives with the second loop. In NVIDIA's own measurements, SWE-bench Verified rises from 50.3 at one loop to 59.5 at two and 61.8 at four, and Terminal-Bench 2.1 from 12.5 to 18.1 to 22.2. The card also claims that more loops reduce the number of generated tokens needed for a task, because the extra work happens internally instead of in a longer written answer, and it advises measuring end-to-end latency and cost on your own workload when choosing.

Looping usually costs memory, because each pass keeps its own key-value cache. Dory offers an optional last-loop caching mode that keeps only the final pass's cache, which NVIDIA says holds memory at the one-loop size for any loop count, with accuracy staying close to the default mode in its experiments. The model was pretrained on 6 trillion tokens, mostly in NVIDIA's 4-bit NVFP4 format before switching to BF16, supports up to 256K tokens of context in English, and is released under the OpenMDW 1.1 license. All benchmark figures are NVIDIA-run under its own evaluation harness.