Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

NVIDIA Snapshot saves a fully started AI inference server, including what it holds in CPU and GPU memory, so that new copies can be restored from that saved state instead of starting from scratch. NVIDIA introduces it as an open-source Kubernetes tool, originally built for NVIDIA's Dynamo inference server, and reports that in its tests a restored replica was serving in 3.6 to 42.7 seconds, 8 to 36 times faster than a normal start. The point for anyone running models is capacity during a traffic spike or after a crash: a free GPU is not useful until a model server on it has finished starting, and for large models that start takes minutes.

The post's breakdown of those minutes explains where the gain comes from. A cold start does four things: launches the engine processes, reads the model weights, allocates memory for the model's working cache, and compiles and tunes GPU kernels and records CUDA graphs (prebuilt sequences of GPU operations that make later requests faster). Loading weights is often not the largest part. For DeepSeek-V4.1-Flash on vLLM across eight GPUs, compilation, tuning and graph capture took 785 of 1,111 seconds, and that configuration showed the largest speedup, 36x. A restore skips all four steps because the process resumes from the point where it was saved. Restoring has costs of its own: CRIU, the standard Linux tool for saving and restoring processes, brings back the CPU-side process, and the GPU memory is copied back separately. For the eight-GPU models, the GPU copy took 18 to 29 seconds, the largest part of the restore.

To use it, an operator asks Snapshot to checkpoint a running Pod (Kubernetes's unit of running containers), or to start one and checkpoint it as soon as it is ready, then starts new Pods with an annotation that names the saved snapshot. The engine has to cooperate with a small pause-and-resume handshake, and the repository includes examples for vLLM, SGLang and TensorRT LLM. Saving is a one-time cost per model and configuration, 15 seconds to 5.5 minutes in NVIDIA's tests, and in every tested configuration saving plus one restore took less time than one cold start. The saved files can be much larger than the weights: gpt-oss-120b on SGLang across four GPUs has 65 GB of weights and produced a 694 GB checkpoint.

The numbers are NVIDIA's own, measured on B200 GPUs with checkpoints stored on a VAST network file system, and both timings exclude pulling the container image and setting up the container runtime, which a real scale-up still pays. Two partner results quoted in the post, from Crusoe and Saturn Cloud on B300 hardware, are also self-reported. A restore needs a node with the same GPU model and driver as the one that made the checkpoint, and the README adds that workloads must fit on one node, that vGPU is unsupported, and that the APIs may still change, so it is not yet recommended for production-critical workloads.

Snapshot is NVIDIA's second recent answer to slow starts. In August, Dynamo's shadow engine recovery cut recovery after a process crash from 283 seconds to 7.3 seconds by keeping a second, already started engine idle on the same GPUs. That approach covers crashes on GPUs that are already serving; Snapshot covers bringing up a replica on a different node. NVIDIA's same-day post on Agent Substrate, a runtime for Kubernetes agents, faces the same restriction to matching GPU model and driver when suspending GPU sandboxes, though neither post links the two projects.