Intel / article

AINews: Megakernels are so dead and so back

Source: latent.space
Author: Latent Space
Date: Published Aug 5, 2026

Context

The megakernel debate concerns which work benefits from combining many GPU operations into one program. The source juxtaposes skepticism about giant kernels for serving models with a successful training implementation. Those are different workloads, so the training result does not settle the broader inference argument. Source discussion.

Cursor built Mixture-of-Kittens for training models, whose specialized components are distributed across GPUs. Data must move between those GPUs as computation proceeds. Its implementation combines communication and computation in one kernel so they can overlap, targeting NVL72 hardware. Cursor reports about a 41% increase in end-to-end training throughput over its earlier DeepEP-based setup in its production training stack. That supports a specific engineering result: integrating data movement with computation helped this training workload. Technical account.

Key takeaways · AI-distilled

The megakernel debate flipped in a day: skeptics argued hand-fused 67k-line kernels never survive production (modular TensorRT-LLM-style kernels win) and that Rubin dependency triggers kill the launch-overhead rationale - then Cursor open-sourced MoK, its NVL72 MoE training megak
Fusion cannot cross tensor-parallel boundaries anyway: nonlinearities like softmax need the full row, forcing inter-GPU communication no matter how much you fuse.
Release wave: Qwen3.8-Max (better and cheaper, 60-80% mAP box-conditioned detection), Pokee-Isaac 28B claiming 10M-token on a single GPU, Maple-Preview 20B ternary weights at 200+ tok/s on a Mac Mini, Mistral Shieldstral 3B on-device safety.
Routing is becoming first-class infrastructure: Not Diamond Code picks model AND reasoning effort per step for 20-65% cost cuts, and Devin Fusion got 4% smarter while 27% cheaper from work alone.
Endpoint quality is now measurable: Artificial Analysis new Endpoint Accuracy Index shows output-token limits and tool-call formatting differences materially degrade serverless endpoints vs self-hosted reference deployments.

Terms in this piece · Glossary

mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.

Key quotes

“i'm sorry that i was right about every single thing.”
ali

“the kernel complexity itself is very difficult to write a very optimized mega kernel.”
Ali

“We checked 12 subreddits, 544 Twitters and no further Discords.”

Read the source www.latent.space