Fast_trimul – Fused Triangle Multiplicative Updates in Python

Fold-model trunks are memory-bound on triangle updates; this kernel folds roughly 1.4× longer sequences on the same GPU behind a one-line patch, and demonstrates CuTe DSL is enough to ship a production kernel without thousands of lines of CUDA.

• articleLiger Kernel Open Source Ecosystem For Efficient Llm TrainingLinkedIn Engineering
• videoCan LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AIAI Engineer
• reponanoAlphaZero – Train a grandmaster-level chess model in 24h with TPUstdoubleu

Checking sign-in…

Loading comments…