Off-chip bandwidth has always been the weak point of wafer-scale.
- Source
- x.com
- Date
Off-chip bandwidth has always been the weak point of wafer-scale. The WSE-3 has staggering on-wafer SRAM bandwidth, but everything slows down the moment data has to leave the wafer. That mattered because the wafer only holds 44GB of SRAM, so big models get spread across many wafers, and every extra wafer means more traffic crossing that starved I/O path. Long context makes it worse, since KV cache balloons the wafer count further. (1/3)🧵 Image Source: Cerebras
CS-4 makes progress here. Off-wafer I/O doubles from 1.2Tb/s to 2.4Tb/s, since the wafer's parallel I/O scales with the 2x clock bump. On top of that sits a new Wafer I/O module: a field-upgradeable FPGA NIC that converts Cerebras's proprietary I/O to standard ethernet. That sounds minor, but it means networking can evolve without a chassis redesign, and it lets the wafer speak whatever the customer's fabric speaks. (2/3)
The bigger story is what this unlocks: heterogeneous disaggregated inference. The wafer is a decode machine, and its rooflines are poor for compute-bound prefill. With the new I/O module, Cerebras can pair with HBM-based XPUs (AMD and Trainium are the announced partners) in both prefill-decode and attention-FFN disaggregated setups. That's the path around the 44GB SRAM ceiling, letting HBM systems hold what the wafer can't. Full Article👇️ (3/3) https://t.co/UV2hYz6d7j
Wafer-scale has been bottlenecked by off-chip I/O and a 44GB SRAM ceiling. Pairing wafers with HBM accelerators for split prefill/decode changes what long- serving on Cerebras can cost and hold.
- KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…