Vibeleaderboard
← All Intel
Intel / video

Talk: Kernels Deep Dive (Ben Burtenshaw)

Source
youtube.com
Author
Hugging Face
Date

Context

A GPU kernel is a small program that does one piece of model math. Ben Burtenshaw of Hugging Face, who says he uses the project but did not write it, explains in this talk that many workloads wait on memory rather than arithmetic: an H100 GPU can do far more math per second than it can read data. Fast kernels therefore combine several steps and read the data once. Flash is his main example.

The pain is distribution. Projects use different layouts and build tools, and he says compiling Flash Attention can take two hours and 96 GB of RAM. Hugging Face's kernel builder enforces one project structure and uses Nix, a tool for reproducible builds, and a Python library fetches the matching prebuilt version with a get-kernel call. He says that cut a Colab build from two hours to about two and a half seconds. That timing is his claim in the talk, not an independent .

Terms in this piece · Glossary
  • attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Read the source www.youtube.com
More from Hugging Face
Recommended reads
Comments

Checking sign-in…

Loading comments…