MiniMax Details Training-Free Low-Bit Attention Speedup for Video Models

MiniMax shared work from partner Nunchux AI applying VC-Attention, a technique for speeding up the attention computation inside its MiniMax-H3 video model without retraining the model itself. Attention is the step where a model compares many pieces of the input against each other; running that comparison in lower-precision numbers makes it faster but can introduce rounding error. MiniMax's post describes two adjustments meant to control that error: V-Smooth, which reduces the error introduced when quantizing one part of the calculation, and ExpCast-FP8, which speeds up the softmax step, the part of attention that turns comparisons into weights, through approximation. MiniMax says Nunchux AI's own evaluation on NVIDIA B200 GPUs found better output fidelity than SageAttention2, an existing low-bit attention method, though this comparison is the partner's self-reported benchmark rather than an independently verified result. MiniMax previously detailed a separate NVIDIA-built speedup for the same H3 model, splitting generation into a low-resolution draft and a refinement pass.