TPUv7 posts first agentic inference benchmark, with perf-per-dollar claims

SemiAnalysis has followed up its report on Google's first TPUv7 submission to AgentX, the agentic inference benchmark in its InferenceX dashboard, with a new thread and a smaller headline number. In the first thread, TPUv7 delivered 1.67 times the performance per dollar of NVIDIA's GB300 NVL72 rack running at FP4. The new thread puts it at 1.42 times and says the lead holds "even on an external non-Google workload cost basis", adding that TPUv7 is 2.5 times better than AMD's MI355X. AgentX, which SemiAnalysis launched in August, is built to mimic agentic workloads, with sessions of many turns and long, growing context.

The attached charts show where the change comes from. Both the earlier and the new charts plot the same Qwen3.5 397B results, dated October 14, under a cost tier labeled owning at large hyperscaler volume, using SemiAnalysis's July 2026 pricing surveys and cost model. The earlier chart priced TPUv7 at $1.03 per chip-hour; the new one prices it at $1.21, while GB300 stays at $2.31 and MI355X is shown at $1.50. Raising the assumed cost of each TPU hour lowers every TPU tokens-per-dollar figure, so the lead narrows even though the measured performance is unchanged. The thread does not explain how the $1.21 figure was derived beyond calling it external and non-Google.

Performance per dollar here is a curve rather than a single number. The charts plot total tokens served per dollar of total cost of ownership against interactivity, the output speed each user sees, measured at the 90th percentile. Serving more users at once lowers the cost per token but slows each user down, so every system traces its own trade-off, and a ratio like 1.42x is read at one point along it. On the new chart, TPUv7 at FP8 sits above GB300 at FP4 through the middle of the range, while GB300 reaches far more tokens per dollar at the slow, high-batch end, about 205 million against roughly 120 million for TPUv7 as read off the chart. FP8 and FP4 are 8-bit and 4-bit number formats, so the Nvidia systems are being compared at the lower precision.

The second chart compares TPUv7 with the older GB200 NVL72, priced at $1.86 per chip-hour. SemiAnalysis says that at medium to high interactivity TPUv7 at FP8 is up to 3.16 times better per dollar than GB200 at FP4, with both using multi-token prediction, which lets a model emit more than one token per step, and with GB200 using disaggregated serving, which splits prompt processing and token generation across separate GPU pools, while TPUv7 used ordinary aggregated serving. At low interactivity GB200 keeps a 1.2x advantage. Both charts still label the TPU result an official preview that may change as validation and publication continue, and it covers a single model; the thread credits the Google TPU, TorchTPU, Inferact, Red Hat and vLLM teams and says more models are coming.