Dylan Patel is the founder and CEO of SemiAnalysis, a research firm covering semiconductor supply chains and AI infrastructure. His reporting connects chip manufacturing, datacenter capacity and the cost of running AI models.[1][2]
Start with Dylan Patel: Three bottlenecks to scaling AI compute →
Links
Institutional biography
X LinkedIn Newsletter Founder and CEO · SemiAnalysis
Career
SemiAnalysis says Patel built the firm from a solo venture beginning in 2020. Its research combines supply-chain reporting with cost models for factories, accelerators and datacenters. His institutional biography distinguishes editing from authorship: most research carries his edit, while much, rather than all, carries his byline.[1][2]
Contributions
Patel co-authored the October 2026 study of AI subscription allowances with Andrew Megalaa and Max Kan. The team treats a subscription as a pool of credits whose consumption changes with model, token type and workload. They infer costs by sending controlled requests and observing changes in usage meters. That approach explains why a subscription cannot be assigned one universal dollar-equivalent value: the result depends on what a customer actually runs, and on limits that providers can change.[3]
Ideas
In his March 2026 Dwarkesh interview, Patel organizes AI-compute constraints around logic chips, memory and power. The conversation traces the businesses that provide these inputs and the economics of labs and cloud providers. It offers a route from an abstract claim that AI needs more compute to the factories, equipment and electricity that determine what can actually be supplied.[4]
Tools, writing & talks
Writing, research & talks
Start here
An interview explaining logic chips, memory, power and the economics of expanding AI infrastructure.
Start with the compute-bottlenecks interview for a verbal map of the supply chain. Its chapter markers let you focus on chips, memory or power before tackling detailed paid research.
Interview guest; hosted by Dwarkesh Patel[4]- Amazon’s Trainium Takes a Strike at the Training Kingnewsletter.semianalysis.com
SemiAnalysis tests Trainium3 racks and finds AWS Neuron now supports native eager-mode PyTorch with torch.compile, making research training viable, while noting bugs and a large maturity gap versus CUDA. Anthropic already runs inference and RL rollouts on it.
Catalog workOriginal source - The Huawei-DeepSeek Symbiosisnewsletter.semianalysis.com
SemiAnalysis examines how Huawei's Ascend 950PR hardware and DeepSeek's kernel software (TileLang, DeepGEMM, FlashMLA, DeepEP) now interlock, arguing DeepSeek will increasingly run on a fully domestic Chinese stack rather than Nvidia compute.
Catalog workOriginal source - Datacenters 2027: The Year of Acceleration, +50GW in a Single Yearnewsletter.semianalysis.com
SemiAnalysis claims over 24GW of datacenters are in advanced construction for 2027 delivery across 40 campuses, supporting a forecast of about 50GW added that year and pushing back on a 25GW skeptic estimate, while noting labor and supply constraints.
Catalog workOriginal source - Anthropic Subscriptions Offer 5x+ More Value Than OpenAInewsletter.semianalysis.com
A measured comparison of subscription credit consumption across models and workloads, published October 2026.
Co-author with Andrew Megalaa and Max Kan[3]Original source - How GLM5.3 Sparse Attention Affects HBM Memory Usagenewsletter.semianalysis.com
SemiAnalysis examines why sparse attention does not shrink memory capacity needs, since top-k selection wants the whole context in HBM, and how SGLang's HiSparse offloads KV cache to host DRAM with LRU caching and layer-wise overlap.
Catalog workOriginal source - The Chinese AI Infrastructure Boom: Introducing the SemiAnalysis China Datacenter Modelnewsletter.semianalysis.com
SemiAnalysis extended its building-level datacenter tracking model to China, finding over 24GW of built AI datacenter capacity, more than EMEA or the rest of Asia-Pacific, challenging assumptions that Chinese AI infrastructure is small or largely vacant.
Catalog workOriginal source - ClusterMAX 3.0: The Industry Standard GPU Cloud Rating System Returnsnewsletter.semianalysis.com
SemiAnalysis publishes ClusterMAX 3.0, a hands-on tested ranking of 77 GPU cloud providers across compute, networking, storage, orchestration, and security, drawn from direct probing of provider infrastructure.
Catalog workOriginal source - Engrams Embedding Entendre: Codesign for Efficient DRAM/SSD Offloadingnewsletter.semianalysis.com
SemiAnalysis explains Engram, a model architecture that extends token embeddings with learned multi-token lookups addressable by token ID, letting those rows prefetch from host DRAM instead of sitting in HBM, cutting HBM needs without hurting quality.
Catalog workOriginal source - Everyone Says Datacenter Moratoriums Are Killing the US Buildout. We disagreenewsletter.semianalysis.com
SemiAnalysis argues the narrative that state and local datacenter moratoriums are halting the US AI buildout is overstated, following its earlier debunking of water-use and cancelled-capacity myths.
Catalog workOriginal source - Vera Rubin NVL72 Agentic Inference: 67x better Performance per Dollarnewsletter.semianalysis.com
SemiAnalysis publishes the first verified agentic-inference benchmarks for Nvidia's Rubin NVL72 platform, finding up to 7x better token throughput per megawatt than Blackwell on early pre-release software.
Catalog workOriginal source - A Brain Too Big to Carry — On-Device vs Datacenter Inferencenewsletter.semianalysis.com
SemiAnalysis argues robotics inverts typical LLM economics: real-time control loops and per-unit hardware costs force robot AI models to be designed around fixed, affordable on-device compute rather than scaled freely like cloud models.
Catalog workOriginal source - Long Live the Short King: Why 4-hi HBM Winsnewsletter.semianalysis.com
SemiAnalysis argues the industry is pivoting from taller 12-hi HBM stacks toward 4-hi and 8-hi configurations, pointing to Nvidia's Rubin Ultra capacity cut to 192GB, and explains why lower stacks offer better bandwidth-per-dollar for inference workloads.
Catalog workOriginal source - What is So Hard About Behind-The-Meter Power For Datacenters? Part 1newsletter.semianalysis.com
SemiAnalysis tracks 75GW of binding behind-the-meter power orders for AI datacenters, sharply up in Q2 2026, and lays out the execution risk between a signed power order and an operating facility.
Catalog workOriginal source - Where Does a Robot Think – On-Device vs Datacenter Inferencenewsletter.semianalysis.com
SemiAnalysis examines why robotics inverts the LLM design pattern: real-time control loops and per-unit cost constraints force robot inference onto the device rather than the datacenter.
Catalog workOriginal source - TPU Inference Externalization Full Steam Ahead - InferenceXnewsletter.semianalysis.com
SemiAnalysis publishes the first third-party benchmarks of Google's TPUv7 Ironwood, finding up to 50% better performance-per-dollar than Nvidia B200/B300 on inference. It compares Google's internal TCO against what external customers actually pay as Anthropic scales toward over a million TPUs.
Catalog workOriginal source SemiAnalysis founder Dylan Patel walks through lab compute, capex trajectories and revenue, arguing the world economy increasingly tracks lab economics and the compute market, with figures out to 2028.
Catalog workOriginal source- Korea’s Trillion-Dollar Sovereign AI Investment: Nvidia Wins, Hynix Losesnewsletter.semianalysis.com
An argument for why governments are funding sovereign AI: frontier model access sits with a handful of US labs and their regulator, releases have been delayed on safety and policy grounds, and open weight licenses now carry revenue thresholds. Korea's buildout is read as an Nvidia win.
Catalog workOriginal source - Most Neoclouds Suck At Securitynewsletter.semianalysis.com
Hands-on testing across GPU neoclouds for ClusterMAX 3.0 turned up five recurring security failure patterns in the providers that AI labs increasingly rent from, alongside an open CLI that audits your own cluster or standalone GPU box.
Catalog workOriginal source Dylan Patel and Dwarkesh Patel on where lab economics are heading: spend rotating back toward training as recursive self-improvement nears, OpenAI and Anthropic outbidding everyone for usable FLOPs, and whether a decade of multi-trillion AI capex ends in a sovereign debt crunch.
Catalog workOriginal source- Are Open Models Catching Up?newsletter.semianalysis.com
An analysis of whether open weights have caught the closed frontier on coding and agentic work, with figures on inference volumes, provider competition and what capable open models do to token pricing.
Catalog workOriginal source - Cerebras's Next Generation CS-4: Fast Just Got Fasternewsletter.semianalysis.com
SemiAnalysis breaks down Cerebras CS-4, which doubles inference throughput on the same 5nm wafer by roughly doubling clock speed, plus the power, cooling and I/O changes that make disaggregated inference possible.
Catalog workOriginal source - $12B of US ratepayers' money wasted on a modeling mistake and PJM wants to do it againnewsletter.semianalysis.com
SemiAnalysis rebuilt PJM's black-box Reserve Requirement Study and argues its methodology ignores winter efficiency gains and post-Elliott resilience, understating capacity by ~4GW. The knock-on: an estimated $12B in avoidable ratepayer cost and a capacity market structurally hostile to load growth.
Catalog workOriginal source - Ultra-High Interactivity on NVIDIA GPUs? - TileRT InferenceXnewsletter.semianalysis.com
Premium fast modes show users will pay for lower latency, so frontier labs are evaluating purpose-built inference hardware that trades batched throughput for interactivity, which is what real-time assistants and full-duplex voice need.
Catalog workOriginal source - SpaceX 10GW in 2027 – Why It’s Real, Will Drive $300B ARR for SpaceX, and Why Microsoft Will Be the Largest Offtakernewsletter.semianalysis.com
SemiAnalysis argues SpaceX's 6-10GW datacenter target for 2027 is achievable, walking the suitable sites and gas-generation equipment that make it possible, then prices scarce near-term capacity at up to $50B per GW per year against the $100B-plus per GW that labs can earn selling API inference.
Catalog workOriginal source - Kimi K3: The Manos, The Mythos, The Legendosnewsletter.semianalysis.com
A SemiAnalysis technical deep-dive into Kimi K3's architecture, tracing Kimi Delta Attention (KDA) from linear attention through DeltaNet and Gated DeltaNet, and detailing Moonshot's FlashKDA kernel implementation with FLOP and memory-traffic complexity analysis. The piece also examines Kimi K3's hybrid KDA/MLA attention design, KV cache throughput and residency across memory tiers, and the challenges linear attention poses for prefix caching in inference engines like vLLM.
Catalog workOriginal source - Can AMD break the CUDA Moat? AMD Advancing AI 2026newsletter.semianalysis.com
SemiAnalysis revisits its AMD call, from giving it a 0% chance of closing the gap with NVIDIA to a materially better one, and assesses whether AMD can break the CUDA moat.
Catalog workOriginal source - Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysisnewsletter.semianalysis.com
SemiAnalysis on Vera Rubin NVL72, the second generation of NVIDIA's rack-scale Oberon architecture, where engineering samples running DeepSeek R1 show 5.4x performance per megawatt and 5x per dollar over GB200 NVL72.
Catalog workOriginal source - The Future of Meta Superintelligence: A 1 Year Progress Updatenewsletter.semianalysis.com
A SemiAnalysis deep-dive argues Meta's Superintelligence Labs (MSL) is uniquely positioned among AI labs to become world-class in data, talent, and compute simultaneously, despite a rocky Llama 4/Muse Spark start. It details Meta's aggressive RL-environment data strategy (including employee screen-recording programs and a new 3,000-engineer applied AI org), massive multi-gigawatt 'titan' datacenter buildouts across five sites, and compute projections showing Meta potentially surpassing OpenAI and Anthropic in total AI compute by year-end.
Catalog workOriginal source - Meta Compute: Everyone Wants To Be A Neocloudnewsletter.semianalysis.com
A SemiAnalysis report arguing Meta's datacenter and compute procurement is accelerating rather than slowing, with over 5GW contracted in just the first half of the year across cloud and colocation deals. It outlines four ways Meta could monetize its massive compute buildout beyond training its own frontier models: scaling ads recommendation systems 10x, striking a Bedrock-style deal to resell private Anthropic Claude access, pursuing short-term high-margin 'SpaceX-style' compute deals like Elon Musk's Anthropic and Google contracts, and building vertical AI-powered sales/marketing products atop its ad platform.
Catalog workOriginal source - ECTC 2026 Advanced Packaging Roundupnewsletter.semianalysis.com
A SemiAnalysis technical report summarizing ECTC 2026 disclosures on AI accelerator packaging, covering Intel's EMIB-T bridge technology (TSVs, MIM capacitors, sub-25μm bump pitch scaling), Marvell's custom HBM4E base-die architecture, Samsung's silicon interposer and hybrid copper bonding thermal work, and emerging microfluidic cooling and photonic interconnect approaches from TSMC, Microsoft, and Lightmatter. It argues that packaging (not transistor scaling) is now the primary bottleneck for AI accelerators, detailing specific bandwidth, power-delivery, and thermal-resistance tradeoffs across vendors.
Catalog workOriginal source - TokenBudgeting: Our Conversations with Enterprises on Token Spendnewsletter.semianalysis.com
Reports say enterprise token consumption has hit a budgeting wall. SemiAnalysis talked to more than 50 customers and argues the widely reported clampdowns at Meta and Uber are overstated and rooted in poor incentives.
Catalog workOriginal source - RL Systems: Mind the Gap — Matching Trainer and Generator Throughputnewsletter.semianalysis.com
A SemiAnalysis technical article arguing that RL training system efficiency hinges on matching trainer (learning) and generator (inference/rollout) throughput, covering GRPO, asynchronous PipelineRL, policy staleness, and RL sandbox infrastructure challenges. It includes case studies training MoE models on agentic coding tasks using open-source RL frameworks, comparing costs against hosted solutions like Thinking Machines' Tinker.
Catalog workOriginal source - Is SMIC N+3's Metal Pitch Smaller than Intel 18A's?newsletter.semianalysis.com
A SemiAnalysis teardown of Huawei's Kirin 9030 (SMIC N+3) compares it against TSMC N6 (via a MediaTek Helio G99) and Intel's 18A, finding that SMIC matches TSMC N6-class logic density only through aggressive DUV multi-patterning and DTCO rather than EUV, at the cost of process maturity and cost efficiency. The report details die floorplans, CPU/GPU/NPU architecture changes versus the Kirin 9020, memory (Samsung and CXMT LPDDR5X) and iPoP packaging, showing Huawei's silicon closing some gaps under export controls while still trailing leading-edge Apple, Qualcomm and MediaTek chips in performance and efficiency.
Catalog workOriginal source - InferenceX DeepSeek v4 Performance Trackernewsletter.semianalysis.com
An open-source SemiAnalysis initiative that benchmarks DeepSeek v4 Pro inference performance across hardware (GB300 NVL72, Huawei Ascend 950DT, MI355X, B200, H200) and engines (vLLM, SGLang, TensorRT-LLM, ATOM) from Day 0 through subsequent weeks, tracking how throughput and interactivity improve as vendors patch and optimize their stacks. The article documents specific bugs (e.g., a hardcoded hidden-size guard in TensorRT-LLM, broken AITER kernels on AMD ROCm) and shows AMD's MI355X achieving over 100x throughput improvement within 26 days via kernel-level fixes.
Catalog workOriginal source SemiAnalysis's Dylan Patel breaks down the semiconductor and power bottlenecks limiting AI compute, how hyperscaler capex converts to gigawatts online, and what lab fundraises cover in rental compute cost.
Catalog workOriginal source- SemiAnalysis newsletternewsletter.semianalysis.com
A research publication following chip production, infrastructure and AI model economics.