Off-chip bandwidth has always been the weak point of wafer-scale.

Off-chip bandwidth has always been the weak point of wafer-scale. The WSE-3 has staggering on-wafer SRAM bandwidth, but everything slows down the moment data has to leave the wafer. That mattered because the wafer only holds 44GB of SRAM, so big models get spread across many wafers, and every extra wafer means more traffic crossing that starved I/O path. Long context makes it worse, since KV cache balloons the wafer count further. (1/3)🧵 Image Source: Cerebras

CS-4 makes progress here. Off-wafer I/O doubles from 1.2Tb/s to 2.4Tb/s, since the wafer's parallel I/O scales with the 2x clock bump. On top of that sits a new Wafer I/O module: a field-upgradeable FPGA NIC that converts Cerebras's proprietary I/O to standard ethernet. That sounds minor, but it means networking can evolve without a chassis redesign, and it lets the wafer speak whatever the customer's fabric speaks. (2/3)

The bigger story is what this unlocks: heterogeneous disaggregated inference. The wafer is a decode machine, and its rooflines are poor for compute-bound prefill. With the new I/O module, Cerebras can pair with HBM-based XPUs (AMD and Trainium are the announced partners) in both prefill-decode and attention-FFN disaggregated setups. That's the path around the 44GB SRAM ceiling, letting HBM systems hold what the wafer can't. Full Article👇️ (3/3) https://t.co/UV2hYz6d7j