DeepSeek V3, SGLang, and the state of Open Model Inference in 2025 (Quantization, MoEs, Pricing)
Source
youtube.com
Author
Latent Space
Date
Why it matters
Covers practical issues in serving a 671B mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → model, including SGLang, quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition → and pricing. Helps you judge the cost and engineering of running open models.
Terms in this piece · Glossary
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
videoThe State of Reasoning — from Nathan Lambert, Interconnects/AI2 [LS Live @ NeurIPS 2024]
video⚡️ARC-AGI-3: The Interactive Reasoning Benchmark
videoThe Agents Economy Backbone - with Emily Glassberg Sands, Head of Data & AI at Stripe
videoOne Year of MCP — with David Soria Parria and AAIF leads from OpenAI, Goose, Linux Foundation
videoDeepSeek, Reasoning Models, and the Future of LLMsa16z
article[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whalewww.latent.space
videoThe Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1Latent Space