To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!

Together AI found that moving its matrix-multiplication code to NVIDIA's Vera Rubin hardware left much of the chip's computing capacity unused. The arithmetic units could consume data faster, but the existing code could not supply it quickly enough. Its ThunderKittens engineering account explains how reusing data already on the chip and preparing more inputs ahead of time addressed that bottleneck.

For a tested 16K-square matrix multiplication using NVFP4 arithmetic, the revised pipeline reached about 22.2 quadrillion floating-point operations per second. That measures one computational building block on qualification-sample hardware, rather than an entire AI application's speed. The result shows why faster chips also need software adapted to their data flow.