Google TPUv7 posts first agentic inference benchmark result

SemiAnalysis says Google has submitted its first results for TPUv7, its seventh-generation AI chip, to AgentX, the agentic inference benchmark SemiAnalysis launched in August, and that they show 1.67 times better performance per dollar than Nvidia's GB300 NVL72 rack (thread). AgentX imitates the traffic a coding agent creates: sessions with tens or hundreds of turns, contexts that grow quickly from system prompts, tool definitions and history, heavy reuse of earlier context because each turn resends everything before it, and bursts of short-lived sub-agents with fresh context. A diagram in the thread shows the server caching the shared earlier portion from one turn to the next, which is why SemiAnalysis treats cache behavior as part of the test.

The test served one open model, Qwen3.5 397B. Google ran it on TPUv7 in FP8, an 8-bit number format, using its TorchTPU build of the vLLM serving engine; the Nvidia system ran in FP4, a 4-bit format, using Dynamo with SGLang. The chart does not give a single score. It plots total tokens served per dollar of total cost of ownership against interactivity, the output speed each user sees at the 90th percentile. Packing more users onto the hardware lowers the cost per token but slows each user down, so every system traces a curve, and the 1.67x label marks the gap at one point on it. SemiAnalysis says TPUv7 FP8, with just four chips, beat GB300 FP4 across most of that curve. At low interactivity GB300 FP4 still wins, peaking on the chart at about 205 million tokens per dollar against about 141 million for TPUv7, although TPUv7 FP8 still beats GB300 running FP8 there.

The dollar side is SemiAnalysis's own estimate, not a market price. The chart's cost tier is owning the hardware at large hyperscaler volume, at $1.03 per TPUv7 chip-hour against $2.31 for GB300, taken from its July 2026 pricing surveys and cloud cost model. The next day SemiAnalysis posted a follow-up putting the advantage at 1.42x on what it called an external, non-Google cost basis, a smaller figure than the 1.67x here. The chart labels the TPU curve an official preview whose results may change as validation and publication continue.

SemiAnalysis frames the result as evidence that Google is pushing TPUs to outside users, a theme of its earlier article on TPU inference, which asked whether an open-weight model served through a familiar inference engine could beat Nvidia on cost. The thread credits Google's TPU and TorchTPU teams along with Inferact, Red Hat and the vLLM project, and says Google will submit many more models and that a full article is coming.