Explains why hand-fused megakernel inference designs are losing favor as new hardware (NVIDIA Rubin) solves the overhead problem they targeted, and flags Cursor's open-sourcing of its MoK training megakernel — relevant for engineers making GPU kernel-fusion tradeoffs.
Key takeaways · AI-distilled
The megakernel debate flipped in a day: skeptics argued hand-fused 67k-line kernels never survive production (modular TensorRT-LLM-style kernels win) and that Rubin dependency triggers kill the launch-overhead rationale - then Cursor open-sourced MoK, its NVL72 mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → training megak
Fusion cannot cross tensor-parallel boundaries anyway: nonlinearities like softmax need the full row, forcing inter-GPU communication no matter how much you fuse.
Release wave: Qwen3.8-Max (better and cheaper, 60-80% mAP box-conditioned detection), Pokee-Isaac 28B claiming 10M-token context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → on a single GPU, Maple-Preview 20B ternary weights at 200+ tok/s on a Mac Mini, Mistral Shieldstral 3B on-device safety.
Routing is becoming first-class infrastructure: Not Diamond Code picks model AND reasoning effort per step for 20-65% cost cuts, and Devin Fusion got 4% smarter while 27% cheaper from agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → work alone.
Endpoint quality is now measurable: Artificial Analysis new Endpoint Accuracy Index shows output-token limits and tool-call formatting differences materially degrade serverless endpoints vs self-hosted reference deployments.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Key quotes
“i'm sorry that i was right about every single thing.”
ali
“the kernel complexity itself is very difficult to write a very optimized mega kernel.”
Ali
“We checked 12 subreddits, 544 Twitters and no further Discords.”
video⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins
videoAnthropic’s Felix Rieseberg on AI Coworkers, Local-First Agents, and the Future of Knowledge Work
videoScaling Test Time Compute to Multi-Agent Civilizations — Noam Brown, OpenAI
videoThe Agent Cloud: Databricks’ Bet on the Future of AI — Matei Zaharia and Reynold Xin
