Vibeleaderboard
← All Intel
Intel / post

In long-horizon stress testing, Muse Code iteratively optimized GPU kernels…

Source
x.com
Date
AI at Meta@AIatMeta

In long-horizon stress testing, Muse Code iteratively optimized GPU kernels over 1,000+ tool calls (up to 24 hours) on @nvidia Hopper GPUs, delivering very competitive performance gains for KDA and MLA relative to baseline Triton implementations.

The baseline is the FLA Triton implementation of KDA. Models were prohibited from importing third-party kernel libraries such as FLA directly; instead, they had to apply specialized kernel-optimization knowledge to implement the algorithm in Triton, rather than wrap existing implementations. Muse Spark 1.2 paired a chunk-parallel preparation kernel with a sequential inter-chunk scan, combining standard fusion and tiling with KDA-specific optimizations such as re-centering the gated cumulative decay at the chunk midpoint.
We benchmark against a PyTorch reference implementation at batch size 1, number of heads 64, sequence length 8192, and latent dimension 512. Muse Spark 1.2 designed a two-kernel Triton pipeline for this workload, combining kernel fusion and tiling with MLA-specific optimizations such as reusing the shared KV latent as both K and V.
View on X

Context

Meta tested whether Muse Code could keep improving GPU kernels, the programs that execute particular calculations on a graphics processor. Its stress-test post describes runs exceeding 1,000 tool calls and lasting up to 24 hours on NVIDIA Hopper hardware.

The release article explains the loop: write code, compile it, measure performance and revise it. The two workloads used different baselines: an existing Triton implementation for KDA and a PyTorch reference for MLA. The reported gains therefore concern those particular comparisons, rather than a general speed increase for GPU software.

Read the source x.com
More from AI at Meta
Recommended reads
Comments

Checking sign-in…

Loading comments…