SemiAnalysis: OpenAI's Jalapeño beats Rubin on perf per watt

If a lab's in-house accelerator genuinely beats Nvidia's current part on performance per watt, the cost floor for inference — and who controls it — starts moving.

• speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

• postOpen models are eating easy tasks. Do frontier labs outgrow it?
• postSemiAnalysis: a $200 Claude plan can return $12,000 in API-priced tokens
• postWhy a subscription plan has no fixed API-equivalent value
• postAI is flooding ML conferences with papers faster than reviewers can scale

• postSemiAnalysis: Jalapeño beats Vera Rubin on throughput per megawattSemiAnalysis
• articleOpenAI Jalapeño: Better Than Nvidia Blackwell OpenAI’s self-designed ASIC…SemiAnalysis
• postSemiAnalysis on Jalapeno: OpenAI's chip, measured in tokens per megawattSemiAnalysis

Checking sign-in…

Loading comments…