Clip / Other

What NVFP4 actually is: microscaling explained

From Compression at the Edge — Chris Alexiuk, NVIDIA · ≈16:47

“the genius isn't it is a microlock scaled uh number.”

“Nvidia adopted like a similar but different design of uh four bits but every uh 16 bits share one uh 8 bit value uh to scale them”

“that has been like that has been tremendous it uh gives significant improvements over like other formats yeah of 4bit.”

What’s in it

Explains what NVFP4 actually is under the hood
Breaks down microscaling: how 16 elements share one scale factor
Traces NVFP4's lineage back to bitsandbytes' quantization trick

Watch this moment

Clip transcript

like you know uh FP8 like what what the hell is NVFP4? >> Oh cool. Okay so FP4 um the number is that it's a floating point 4bit number but the genius isn't it is a microlock scaled uh number. So microlock scaling means uh every u you choose a group. In in the case of FP4, you choose 16 elements and you can share one scaling fac one one extra um FP8 number between these 16 elements. So this was originally u invented by uh uh bits and bites Tim that yeah so uh Nvidia adopted like a similar but different design of uh four bits but every uh 16 bits share one uh 8 bit value uh to scale them and yeah that has been like that has been tremendous it uh gives significant improvements over like other formats yeah of 4bit. Yeah.