Intel / article

Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes

Source: developer.nvidia.com
Author: Shara Tibken
Date: Published Oct 22, 2026

Context

The main thing an eight-node cluster adds is memory. In this NVIDIA developer post, NVIDIA says each DGX Spark desktop machine has 128 GB of unified memory, so four linked nodes reach 512 GB and eight reach 1 TB. Official eight-node support, announced at GTC Berlin, lets models that would not start on four nodes run locally, including the full 753-billion-parameter GLM 5.3 from Z.ai. On that model NVIDIA reports about 31 per second of decode for a single user, falling to 18 per user (67 in total) with eight concurrent users. Eight-node support in NVIDIA Sync, its setup tool, is due by the end of the year.

Adding nodes helps generation speed, but not in proportion. Splitting a model across machines means they must constantly combine partial results, an operation called all-reduce. On a dense Qwen3.8-27B model, decode rose from about 12 tokens per second on one node to 52.58 on eight switch-connected nodes, a 4.43x gain, and 40.11 (3.38x) on eight nodes wired in a ring. Part of the switch result comes from RoCEnante, a new backend now merged into FlashInfer. Instead of passing data around a ring over several rounds, each node sends its input directly to every other node. That cuts latency for the small messages typical of decode, but moves more data. NVIDIA says the standard NCCL library remains the default and does better with more nodes, high concurrency or ring wiring.

The wiring choice matters. NVIDIA presents a switchless ring as the cheaper, simpler option for a four-node box, and recommends a switch for scaling to eight nodes. A switch is required to split the cluster into independent groups or to use RoCEnante. One use for splitting is NeMo Switchyard, which routes easy requests to a small model and hard ones to a larger one. In NVIDIA's prototype, the routed setup scored 91 out of 100 on a 92-case tool-use , matching the large model alone. It raised total throughput about 1.5x when half the requests went to the large model and about 3x when a quarter did.

Training scales much better than inference here, because each node holds a full copy of the model and only synchronizes gradients once per step. an 896.5-million-parameter nanochat model ran 7.79x faster on eight nodes than on one, 97.4% scaling efficiency. All of these figures are NVIDIA's own benchmarks.

Key takeaways · AI-distilled

Eight-node DGX Spark clusters reach 1 TB of unified memory (128 GB per node), enough for the full 753B GLM 5.3. NVIDIA measured roughly 80 tokens per second of generation with GLM 5.3 Flash on eight nodes.
On dense Qwen3.8-27B (NVFP4), decode scaled from 11.86 tokens/s on one node to 52.58 on eight switch-connected nodes (4.43x), versus 40.11 (3.38x) on an eight-node physical ring. Prefill scaled only 1.94x.
RoCEnante replaces ring all-reduce with one all-to-all exchange plus local reduction, cutting communication steps for small tensor-parallel decode messages at the cost of more traffic. It needs a switch, falls back to NCCL, and NCCL stays better as scale or concurrency grows.
A switchless four-node ring is the cheapest option and matches NCCL's default ring algorithms, but splitting the cluster into groups, MPI-style all-to-all work, and the RoCEnante backend all require a switch.
With NeMo Switchyard routing easy requests to MiniCPM5 2B and hard ones to Qwen3.8 Flash Next, quality matched the Qwen-only setup (91/100 on a 92-case tool-use benchmark) and aggregate throughput rose about 3x when a quarter of requests went to Qwen.

Terms in this piece · Glossary

token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.

Read the source developer.nvidia.com