Wafer expands its open GPU performance engineering…

Context

Wafer's AI Performance Engineering guide is an MIT-licensed reading list on GitHub. It teaches how to make AI models run fast on GPUs, with the material arranged in the order the problems appear (GitHub). A short "Start here" section covers the minimum mental model. It follows one request through prefill and decode, explains the roofline model, which shows whether a computation is limited by arithmetic or by memory bandwidth, and works out what costs in FLOPs, bytes and communication. The list then moves through GPU fundamentals, kernel optimization, programming models and profiling, inference engines, distributed inference and current hardware. Wafer suggests reading it in order if you are new and otherwise using it as a reference.

The guide is defined less by its topics than by its admission rule. A core entry must be the paper that introduced a mechanism, the specification or official documentation that defines it, the repository that implements it, or a direct implementer report with code and measurements. Performance numbers are omitted unless they come with hardware, workload, precision, baseline and correctness method. The guide also says vendor peak numbers are not performance measurements. Faster-moving material, such as AI-generated kernels and a watchlist of unreleased hardware, sits in a separate Frontier section, last verified on October 17.

Wafer describes this update as a large expansion and names four additions (Wafer on X). Stream-K deals with matrix multiplications in which the final batch of output tiles leaves part of the GPU idle. Its authors divide the inner-loop work evenly across processors instead of assigning whole tiles. In tcgen05 for dummies, gau-nernst builds a Blackwell matrix multiply step by step. Thinking Machines' Defeating Nondeterminism explains how a change in batch size can change the order of floating-point additions, and therefore the model's output, and how to write batch-invariant kernels. Triton's Linear Layouts represents how tensor elements are spread across threads and registers as binary matrices, so the compiler can derive the data movement needed to rearrange them. Wafer says the update also expands the , and hardware references.

Wafer was already promoting the repository in late September, in a thread series walking through individual entries. Only the current README is available here, not a diff, so the list of additions comes from Wafer's thread. The linked papers were read as abstracts at most. This explanation therefore covers how the guide is built and what it chooses to include, not the results claimed by any of its sources.

Key takeaways · AI-distilled

Wafer orders the guide from a single inference request outward, through GPU fundamentals, kernels, programming models, inference engines and distributed serving, and asks newcomers to read an eight-item Start here set first.
Its source policy admits only the paper that introduced a mechanism, the defining spec or docs, the implementing repository, or a reproducible implementer report, and drops any performance number lacking hardware, workload, precision, baseline and correctness method.
A separate Frontier section, verified 2026-10-17, tracks AI-generated kernel work: benchmarks such as KernelBench-Verified and FlashInfer-Bench, plus KernelGuard, which layers static rules, physics-floor checks and an LLM judge against reward-hacked kernels.
NVIDIA Rubin, AMD MI455X, Google TPU 8t and 8i, and UALink 2.0 stay on a watchlist pending reproducible measurements on shipped or generally available systems; the guide states vendor peak numbers are not performance measurements.

Terms in this piece · Glossary

transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.