Activation checkpointing burns FLOPs you never recover; on Grace Blackwell, offloading MLP activations over C2C replaces it for a 6–13% throughput gain at roughly 0.5% more peak memory.

• articleLong Context Update Laguna Xs 2 And M 1
• articleIntroducing The Poolside Platform
• articleIntroducing The Model Factory
• articleThrough The Looking Glass

• articleReducing High-Bandwidth Memory Bottlenecks in JAX-Based LLM Training with Host OffloadingTanya Lenz
• articleFlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware ScalingTogether AI
• videoCan LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AIAI Engineer

Checking sign-in…

Loading comments…