Vibeleaderboard
← All Intel
Intel / post

How does it compare to Qwen 3.6 35B A3B, one of the most popular models on DGX S

Source
x.com
Date
Alex Cheema@alexocheema
Thread · 2 parts

We worked with @nvidia to understand how Nemotron 3.5 Lightning performs locally on the DGX Spark. We ran the same set of 884 agent tasks on 100 unique models and compared them on Intelligence and Speed. Here's what we found 👇 (Full results on https://t.co/b6s9nCUDko) https://t.co/RD0jmQJR8C

How does it compare to Qwen 3.6 35B A3B, one of the most popular models on DGX Spark today? Qwen with MTP actually runs at faster decode TPS than Lightning for most context lengths and tasks. However, Qwen is much more verbose: it uses 45% more output tokens and 11% more steps to complete tasks. This histogram shows the output token distribution over all GDPVal tasks. Overall tasks take 42% longer with Qwen than Lightning.

Context

Alex Cheema's comparison on NVIDIA's DGX Spark shows that a model which generates text faster can still finish tasks later. In the comparison post, Cheema says Qwen 3.6 35B A3B from Qwen, one of the most popular models on the Spark, runs with MTP at a faster decode speed, measured in per second, than NVIDIA's Nemotron 3.5 Lightning for most lengths and tasks. Yet tasks take 42% longer overall with Qwen.

The difference is how much each model writes. Qwen uses 45% more output tokens and 11% more steps to complete tasks, and that extra work outweighs its per-token speed. The comparison is part of a thread in which Cheema's team ran the same set of 884 agent tasks on 100 models (thread opening).

Another post in the thread shows the reverse case: against Step 3.7 Flash, Lightning wrote about as many tokens but generated them faster. Together the two comparisons show that the wait for an agent depends on both how fast a model writes and how much it writes, so a tokens-per-second figure alone does not predict how long a task takes on this machine.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
More from Alex Cheema
Recommended reads
Comments

Checking sign-in…

Loading comments…