Shows that NVIDIA wins on per-query latency at low concurrency while AMD MI300X delivers higher peak throughput at high concurrency. That helps when choosing hardware for serving open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → models.
Key takeaways · AI-distilled
In AMD-commissioned Artificial Analysis tests, at 16 concurrent DeepSeek R1 queries the 8xH200 system hit ~45 median output tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →/s per query and ~600 tokens/s system throughput, versus ~35 and ~500 for 8xMI300X.
On Llama 4 Maverick at 16 concurrent queries, the 8xH100 system reached ~90 median output tokens/s per query and ~2,150 tokens/s system throughput, versus ~70 and ~2,100 for 8xMI300X.
Under ~250 concurrent queries, NVIDIA systems had marginally faster average per-query output speeds, lower average end-to-end latencies and higher throughput than MI300X; MI300X reached 25-35% higher peak system output throughput.
MI300X kept near-perfect response rates (>0 output tokens within the 3-minute window) up to its measured peak throughput, while H200 rates degraded as it scaled to peak; AA says MI300X had better end-to-end latency at higher workloads.
Artificial Analysis says cost per token can be calculated as a cost metric divided by system throughput; on that basis NVIDIA had a slightly lower implied cost per token at low concurrency and MI300X a lower cost at higher concurrency.
Terms in this piece · Glossary
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
