Wafer expands its open GPU performance engineering guide
Source
github.com
Author
wafer
Date
Why it matters
A structured, primary-source reading path covers GPU kernels (Stream-K, Blackwell tcgen05, batch-invariant inference), quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition →, speculative decodingA speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.Full definition →, and distributed serving. It helps engineers working on inference cost and latency find the original papers and specs quickly.
Key takeaways · AI-distilled
Wafer orders the guide from a single inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → request outward, through GPU fundamentals, kernels, programming models, inference engines and distributed serving, and asks newcomers to read an eight-item Start here set first.
Its source policy admits only the paper that introduced a mechanism, the defining spec or docs, the implementing repository, or a reproducible implementer report, and drops any performance number lacking hardware, workload, precision, baseline and correctness method.
A separate Frontier section, verified 2026-10-17, tracks AI-generated kernel work: benchmarks such as KernelBench-Verified and FlashInfer-Bench, plus KernelGuard, which layers static rules, physics-floor checks and an LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → judge against reward-hacked kernels.
NVIDIA Rubin, AMD MI455X, Google TPU 8t and 8i, and UALink 2.0 stay on a watchlist pending reproducible measurements on shipped or generally available systems; the guide states vendor peak numbers are not performance measurements.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
