Intel / post

Excited to collab with @ashxhart on this!

Source: x.com
Date: Published Aug 19, 2026

Excited to collab with @ashxhart on this! Why does prefill/decode disaggregation work on DGX Spark + Mac ? DGX Spark: ~350 TFLOPS@fp4, 128GB@273GB/s M5 Max MacBook Pro: ~70 TFLOPS@fp4, 128GB@614GB/s Prefill is compute-bound. Prefill on Spark runs ~5x faster than M5 Max. Decode is memory-bound. Decode on M5 Max runs ~2x faster than Spark. Simple idea: run prefill on Spark, decode on M5 Max. That's faster than running on one device alone, since each phase runs on the best device for that phase. This requires sending the KV cache from the Spark to the Mac. With a 10 Gigabit connection, we can overlap fully computation and communication of the KV cache. This doesn't require low latency, and works just fine over TCP/IP. RDMA on the other hand enables low latency networking (~single digit microseconds), which is the unlock for scaling up decode with ~equal memory bw devices e.g. DGX Spark + M4 Pro / M5 Pro. Low-latency is also important for DGX Spark + RTX or Mac + RTX. Small pool of very high speed memory + large pool of slower memory is the ideal local hardware architecture. Idea is to put the attention part of the model on the faster small memory pool, and the MoE part on the…

Aug 19, 2026 View on X

Read the full post on X

Context

Alex Cheema proposes dividing local model between an NVIDIA DGX Spark and an M5 Max MacBook Pro. The split follows two different kinds of work: prefill processes the prompt, while decoding generates the continuing response. These phases can benefit from different hardware, as explained in NVIDIA's account of disaggregated inference.

Cheema reports faster prefill on Spark and faster decoding on the Mac, motivating that allocation. The handoff requires transferring the cached state, known as the . Cheema says a 10-gigabit connection can overlap that transfer with computation in this setup. The benefit therefore depends on both the phase-specific advantage and the cost of moving state between machines; the post supplies no complete workload .

Terms in this piece · Glossary

inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.

Read the source x.com