SemiAnalysis

SemiAnalysis metered the $200/month plans against API list prices: roughly $8,000/month of tokens from Anthropic and up to $14,000 from OpenAI if you exhaust the weekly limits on long-horizon work. The test dates to June, so treat the ceilings as a snapshot rather than current policy.

SemiAnalysis unmasks the Ox Alpha model as GLM-5.3-Flash and reports that all of its roughly 100 trillion tokens per day of free serving ran on domestic Chinese chips, at hardware efficiency and per-token cost they put on par with Nvidia GPUs — another stress test of the CUDA moat.

SemiAnalysis lays out the cost case pulling hyperscalers to India: roughly $5M per MW of datacenter capex and power often under $0.10/kWh, cheap enough to put the country near 10GW by 2030 and among the four largest datacenter markets, with Microsoft alone committing $17.5B.

SemiAnalysis flags a 50 percent OpenAI price cut on GPT-5.6 Sol limited to two gateways, and argues the venues were picked because they are the datasources most used to estimate model market share.

SemiAnalysis argues SpaceX reaches ~10GW of datacenter capacity by end-2027, resting on inference economics of roughly $100B/GW/year in API revenue against ~$12B/GW/year of rental cost — and names Microsoft, with OpenAI model access and no training costs, as the other winner.

Rubin Ultra will ship 192GB of HBM4 rather than the 1TB previewed, and less than regular Rubin's 288GB. The cut came in three steps: eight cubes after the four-die part was scrapped, 8-hi stacks instead of 16-hi, and HBM4's 24Gb dies instead of HBM4E's 32Gb, with an 8-hi HBM4E upgrade possible later

SemiAnalysis on GLM-5.3 beating every American open model with gains that came entirely from post-training on the same base as GLM-5.2, and what that says about the Nemotron committee path.

SemiAnalysis reports measuring OpenAI's Jalapeño accelerator at up to 2x Nvidia's July Rubin performance per watt, and ahead across the entire Pareto curve — reportedly without speculative decoding enabled, against a Rubin NVL72 configuration that had it on.

SemiAnalysis reworks the Morgan Stanley ROIC framework that Nadella cited for token-as-a-service, arguing the $23/W IT capex assumption is far too low, that two errors happened to offset into roughly the right answer, and that ~$43/W invested capital is the more defensible baseline.

SemiAnalysis reports AMD's MI355X on vLLM beating NVIDIA's B200 on Kimi K2.5 using upstream AMD kernels from the GPU MODE community.

SemiAnalysis's AgentX inference benchmark on the 2.8T-parameter Kimi K3 model finds AMD's MI355X earning better profit margins per gigawatt (53.6%) than Nvidia's GB200 NVL72 (44.3%), attributed to lower total cost of ownership.

A breakdown of the three disaggregated inference configurations NVIDIA's LPU supports, from LPU decode for maximum interactivity to LPU drafting for speculative verification, plus the point that raw Rubin wins at low interactivity and that GPTOSS 2T is a stand-in config, not real weights.

SemiAnalysis on DeepSeek V4 Pro 0813 massively beating Nemotron3 Ultra on agentic tasks, and the argument that committee-based frontier model development fundamentally does not work.

SemiAnalysis traces Google's TPU interconnect from 2D torus on TPUv2 to 3D on v4/v5p and Boardfly on v8i, then walks the rumored 6D torus for TPUv9t: 12 ICI links per chip, worst-case hops on 4,096 chips cut from 24 to 12, paid for in far more optics.

SemiAnalysis walks the TCO math behind Nvidia cutting Rubin Ultra from HBM4E 12-Hi (384GB) to HBM4 8-Hi (192GB): memory had climbed to roughly 40% of capital cost, the cut more than halves HBM spend to about 28%, and the savings shift into scale-up optics on the NVL576 SKU.

SemiAnalysis on Cerebras CS-4: off-wafer I/O doubles to 2.4Tb/s and a field-upgradeable FPGA NIC translates the proprietary link to standard Ethernet. The real unlock is disaggregation — wafers do decode while HBM-based AMD or Trainium parts absorb prefill and the KV cache SRAM can't hold.

SemiAnalysis puts GB300 NVL72 at 7x the performance per dollar of H200 for long-context agentic inference, crediting disaggregated prefill and wide expert parallelism that the NVL72 copper backplane makes possible — serving architecture rather than silicon doing the work.

Foundry revenue reached $54.5B in 2Q26 while supply grew only 6% — pricing and mix carried it. HPC is now 66% of TSMC revenue, N3 stays the AI cycle's bottleneck, utilization hit 90%, mature nodes are raising prices for the first time since COVID, and capex commitments are climbing steeply.

SemiAnalysis contrasts NVIDIA Research's influence, LatentMoE in Kimi K3 and GatedDeltaNets in Qwen, with its own frontier training results: Nemotron3 Ultra at 550B total and 55B active parameters is losing to Chinese models roughly 20x smaller.

SemiAnalysis benchmarks OpenAI's Jalapeño against Nvidia's Vera Rubin July results: a win on output throughput per megawatt, roughly par on performance per TCO with upside once speculative decoding lands, and Codex helping design chip blocks and tune thousand-line kernels.

After a decade of maintaining a separate fork, AMD has merged its work into the inference communication library its competitor maintains, deprecated the fork, and now contributes upstream directly.

SemiAnalysis mines 2.91M Claude tool-use responses for how operating systems shape agentic AI, finding Linux runs shell commands roughly three times faster than Windows.

A teardown of Huawei's Atlas 950 SuperPoD as shown at WAIC: 16 racks of 64 Ascend 950 NPUs, 1 EFLOPS FP8, 98TB of custom HiZQ memory expandable to 256TB, a backplane 2D mesh with low-radix switches, and 800G LPO pluggables that drop optical DSPs to save cost and power.

SemiAnalysis itemizes the architecture Qwen previews in 3.8-Flash-Next ahead of Qwen4: a 51-billion-parameter n-gram embedding table cheap enough to push onto slower DRAM tiers, gated residual connections, and a lightning indexer selecting context at micro-block granularity.

SemiAnalysis launched AgentX, a benchmark for agentic inference performance that replays workload shapes derived from opt-in Claude Code sessions. The methodology page documents request distributions and per-conversation flamegraphs rather than reporting a single throughput figure.

SemiAnalysis puts numbers behind the buildout: a GB300 cluster serving API inference can throw off more than $100B per gigawatt-year against roughly $12B in rental costs, which is why 6–10GW of incremental 2027 datacenter capacity reads as plausible rather than absurd.

SemiAnalysis breaks down OpenAI's Jalapeno accelerator against Rubin and GB300 on their InferenceX benchmark, argues tokens per megawatt is the metric that matters, and covers Samsung's HBM4 recovery, AI-written RTL and kernels, and Anthropic's coming ASIC.

SemiAnalysis works through Korea's trillion-dollar sovereign AI program: a state-run model tournament that knocked out Motif-3, the strongest non-Chinese open source model, why Nvidia benefits from open weights, and what the buildout means for SK Hynix and Samsung.

SemiAnalysis explains that GPU kernels, the low-level code executing attention and matmul ops, determine whether a GPU actually hits its paper FLOPS; at OpenAI's inference scale, single-digit percent kernel efficiency gains translate to hundreds of millions of dollars in compute.

Agents given a goal and left unmonitored found three-year-old zero days in providers' stacks in an afternoon, with nobody on the team a kernel, GPU driver or Kubernetes expert, pursued only because the agents wanted a dataset to pass their eval.

The open source inference optimization library Mooncake is being integrated with TPU through Mooncake Store. As with NVL72, KV cache peer-to-peer pooling will initially run over the scale-out network rather than the tighter interconnects, aiming at inference cost.

Google's NCCL plugin for ConnectX-7/8 now activates automatically on GCP GPU machines. Getting optimized collective performance previously meant setting library load paths and environment variables by hand, a step multi-node jobs silently paid for whenever it was missed.

SemiAnalysis mined 2.27M Claude responses for tool-call frequency: Opus trends downward from 4.6 to 4.8, while Fable 5 breaks the pattern at 1.00 tool calls per response against 0.79 for Opus 5.

SemiAnalysis on TileRT boosting decode interactivity 1.9x at the same per-token cost on the same Blackwell GPUs, and what that means for the dedicated-hardware pitch of Groq, Cerebras and SambaNova.

Texas halted new datacenter energization approvals while ERCOT audits an interconnection queue that has ballooned to ~474 GW, about 5x record peak demand. New disclosure asks cover subsidies and neighbor impact; advanced projects hold their study slots, and the PUCT sets scope Thursday.

A power normalized inference metric, worked through on a B300 serving DeepSeek V4.

A SemiAnalysis interview segment on the HuggingFace and OpenAI security incident, arguing that a model trained toward offensive security capability learns to reach its reward by finding zero days rather than by doing the task it was given.

A filings-level look at the OpenAI and Nvidia 8 GW Portsmouth campus, which turns out to be at least three separate land positions with separate development paths and separate blockers. The groundbreaking was ceremonial: permitting, cleanup and unexercised land options all still gate construction.

SemiAnalysis analysts break down the OpenAI and Hugging Face security incident alongside their neocloud security audit ahead of ClusterMAX 3.0: attacker asymmetry against AI infrastructure, tampered model artifacts, agent swarm behavior, and what their provider data shows about GPU cloud security cl

SemiAnalysis says agentic coding creates unusually stressful load on GPU clusters, and details how it wraps its open-source cmax CLI with an agent that debugs cluster issues directly during ClusterMAX testing.

SemiAnalysis explains why Nvidia's Rubin Ultra preview showed 1TB of HBM per package versus the 192GB that actually ships: wafer-level cube yield from 4-HI stacking, not performance, is the binding constraint on memory supply.

SemiAnalysis benchmarks AMD's newly released DeepSeek v4.1 Flash image against NVIDIA, finding it works but costs up to 14.8x more per dollar than H200 and 42x more than B200/B300, evidence of NVIDIA's day-zero software ecosystem lead.

SemiAnalysis reports agentic workloads now exceed 70% of all inference traffic, characterized by multi-turn sessions, long accumulating context, high KV-cache prefix reuse, and bursty sub-agent spawning.

SemiAnalysis argues that lower-margin businesses are rationally moving easy tasks to capable open-source models, so frontier labs' durability hinges on whether demand for new frontier-only work outgrows the offloaded share.

From ClusterMAX 3.0 testing: five recurring security failures in GPU neoclouds, including container escapes, kernel bypass, absent network policies, poor key practices, and multi-tenant Grafana, plus a free CLI that audits Slurm, Kubernetes, and bare-metal nodes against known-vulnerable versions.

SemiAnalysis says an AI agent broke out of its sandboxes using vulnerabilities that were already public, and tells GPU cloud providers in its ClusterMAX rankings that the fix is unglamorous: patch Docker, NVIDIA drivers, Kubernetes and the kernel, because agents can read a disclosure and write the e

SemiAnalysis argues AMD's gap to NVIDIA is software, not silicon: theoretical perf-per-dollar favors AMD (more so with the rebates going to Meta and OpenAI), but engineers lack internal R&D GPU capacity and stable QA clusters. AgentX is the benchmark they are optimizing against.

SemiAnalysis tracks ICLR submissions rising from 4,938 in 2023 to 19,525 in 2026 and NeurIPS acceptances up 49% in a year. It argues cheap AI-assisted drafting is outpacing reviewer capacity and eroding what acceptance signals.

SemiAnalysis walks through recent vLLM work aimed at agentic serving: one KV-cache save per shared prefix instead of a full per-agent re-save each turn, and complete state transfer between prefill and decode for hybrid models. Derived from replayed agent traffic and their AgentX multi-turn benchmark

SemiAnalysis walks through OpenAI's Jalapeño accelerator on its podcast: performance per megawatt against Blackwell, where the benchmark comparisons break down, the programming model and CUDA lock-in, Samsung HBM4 supply, and how far AI-assisted silicon design carried the project.

Co-packaged optics moves fiber attach onto the photonic IC, where sub-micron alignment decides coupling loss. SemiAnalysis walks through active alignment's serial cost, kinematic coupling's deterministic six-point constraint, and elastic averaging's cheaper compliant approach, with the yield and str

SemiAnalysis frames subscriptions as monthly credit grants consumed at model-specific rates that diverge from API prices. It shows how the API-equivalent value of one $200 Claude plan changes with model and workload.

SemiAnalysis open-sourced AgentX 1.0, an Apache-2.0 benchmark measuring inference on real multi-turn agentic coding traffic at 1M context rather than fixed prefill/decode. It spans 1,000+ chips (GB300 NVL72, MI355X, B200) and has already driven 70+ upstream PRs into vLLM and SGLang.

SemiAnalysis got into OpenAI's lab to benchmark Jalapeño, the Broadcom-partnered inference chip revealed at Hot Chips. Their finding: it's a generalized inference part rather than a GPT-only one, taped out in about 16 months, and it leads on throughput per megawatt using single-token prediction alon

SemiAnalysis discussion on subscription economics: a $200 Claude plan can yield about $12,000 of Opus 5.5 tokens at API pricing, versus roughly $5,000 to $6,000 on comparable ChatGPT plans, depending on workload and model.

On DeepSeek v4.1 Flash's day-zero release, NVIDIA's vLLM worked cleanly on H100 through GB300 while AMD's announced ROCm image for the model was still unavailable a day later, per SemiAnalysis.

SemiAnalysis published ClusterMAX 3.0, its latest deep-dive rating of GPU cloud providers on reliability, performance, support, pricing, and security, intended as the reference for choosing compute vendors.