Intel
Page 33
Pushing software engineering limits with “napkin math”
Gergely Orosz on napkin math, the practice of reasoning about system limits from a handful of memorised latency and throughput numbers, following an interview with turbopuffer's Simon Eskildsen.
Why it mattersLearn to use 'napkin math' — rough first-principles estimates of compute, I/O, and memory limits — to quickly spot why AI-era systems run slow or cost too much, illustrated through the origins of vector search infra like turbopuffer.
articleGergely Orosz
Nativ: Run AI models locally on your Mac
Nativ wraps MLX in a full macOS desktop app, giving local models a chat interface and a localhost API server, from the developer behind the MLX-VLM library.
Why it mattersNativ gives Mac users a native MLX-backed desktop app for running local LLMs and VLMs, with a chat UI plus an OpenAI-compatible localhost API server.
articleSimon Willison
A Fireside Chat with Cat and Thariq from the Claude Code team
An edited transcript of a fireside chat with Cat Wu and Thariq Shihipar of Anthropic's Claude Code team.
Why it mattersPractical prompting guidance from the people who ship Claude Code — that adding examples and long 'don't do X' lists now hurt results on frontier models, and that their system prompt shrank 80%.
articleSimon Willison
What does 99.9% uptime mean for inference?
Reliability numbers are easy to publish.
Why it mattersIf you depend on an inference provider, this breaks down what each uptime tier really guarantees and the failure domains it must survive, giving you concrete questions to ask before committing to an SLA.
articleTogether AI
zed v1.11.3
Zed v1.11.3 adds dedicated staged and unstaged changes multibuffers for reviewing and staging individual hunks, Git Graph improvements with Markdown commit messages.
Why it mattersZed's ACP integration and hunk-level Git review make it a fast, native editor for developers who want an agent panel and granular version control in one Rust-based tool.
repozed-zippy[bot]
ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)
ParallelKernelBench tests whether LLMs can write fast multi-GPU CUDA kernels across 87 real workloads.
Why it mattersIf you're evaluating whether to trust LLMs to generate performance-critical GPU code, ParallelKernelBench gives you a grounded reality check.
articleTogether AI
Dwarkesh Patel
The data black hole at the center of AI
podcast
Listen
reporhys-firecrawl
firecrawl v2.11.0 — Firecrawl v2.11.0
Firecrawl v2.11.0 adds a research index over 3M+ arXiv papers and the GitHub code behind them, refreshed daily.
Why it mattersFirecrawl's v2.11 adds an arXiv+GitHub research index with SOTA arXivQA recall, keyless MCP/CLI/SDK access.
Kimi K2.7 Code vs Claude Fable 5: Landing pages that cost 94% less
We generated 12 landing pages with Kimi K2.7 Code and Claude Fable 5. Kimi cost 94% less and scored within a few points on every page. Here's what actually moved the needle.
Why it mattersIf you're picking a model for AI-generated landing pages or frontend work, this head-to-head shows Kimi K2.7 Code landing within a few points of Claude Fable 5 quality at ~94% lower cost.
articleTogether AI
articleTogether AI
Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets
How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway.
Why it mattersIf you're serving long-context or multimodal LLMs at scale, this breaks down concrete serving optimizations — KV-block-major sparse attention, paged MSA decode, and a Rust multimodal gateway.
Interconnects: Open and closed models are on different exponentials
An analysis essay from the Interconnects AI newsletter arguing that closed frontier labs (OpenAI, Anthropic) and open model builders are on divergent economic trajectories.
Why it mattersThe case that open and closed models sit on different capability exponentials — and what that means for betting on either.
articleNathan Lambert
How Together AI built the world’s fastest speech-to-text stack
Together AI built the fastest speech-to-text stack on Artificial Analysis by treating ASR as a full-path systems problem, not just a GPU inference problem.
Why it mattersIf you're serving speech-to-text at scale, this breaks down how treating ASR as an end-to-end systems problem — not just GPU kernel tuning — produces state-of-the-art latency and throughput on Artificial Analysis benchmarks.
articleTogether AI
Serving DeepSeek-V4: why million-token context is an inference systems problem
DeepSeek-V4 makes million-token context a serving-systems problem.
Why it mattersIf you're serving million-token-context workloads, this breaks down the real bottlenecks — compressed KV cache layouts, prefix caching, and kernel maturity on HGX B200.
articleTogether AI
Accelerate RL rollouts by up to 50% with distribution-aware speculative decoding
Rollout is the silent bottleneck in RL post-training. DAS fixes it with adaptive speculative decoding — up to 50% faster, zero degradation in reward quality.
Why it mattersIf you're doing RL post-training, rollout generation dominates wall-clock time.
articleTogether AI
Parcae: Doing more with fewer parameters using stable looped models
Parcae is a stable looped language model that matches the quality of a Transformer twice its size — a 770M model reaching 1.3B-level performance.
Why it mattersParcae shows that adding recurrence (looping a smaller model) can match a Transformer twice its size, offering a compute- and memory-efficient path to quality — useful if you're weighing model size against inference cost.
articleTogether AI
EinsteinArena: Harnessing the collective intelligence of agents in the wild to advance science
EinsteinArena is a platform where AI agents collaborate and compete on open math problems.
Why it mattersEinsteinArena demonstrates that competitive/collaborative multi-agent setups can push the frontier on genuinely open math problems, not just closed benchmarks.
articleTogether AI
AI for Systems: Using LLMs to Optimize Database Query Execution
New research shows LLMs can optimize database query execution plans—achieving up to 4.78x speedups by correcting the cardinality estimation errors that statistical heuristics miss.
Why it mattersIf you build data-intensive systems, this shows LLMs can outperform traditional statistical heuristics on cardinality estimation—the classic weak point in query planners.
articleTogether AI
Inside the Together AI kernels team
The team behind FlashAttention and ThunderKittens — how Together AI's kernel researchers close the gap between GPU hardware and production AI.
Why it mattersFlashAttention and ThunderKittens are foundational to modern LLM inference and training performance, so understanding how this team writes GPU kernels gives engineers insight into where real throughput gains come from.
articleTogether AI
Aurora
1.25x over a well-trained static speculator.
Why it mattersIf you're serving LLMs and using speculative decoding, Aurora replaces one-time offline draft-model training with an RL loop that adapts the speculator to your live request distribution, reportedly yielding 1.25x throughput over a well-tuned static speculator.
articleTogether AI
Plan, divide, and conquer: How weak models excel at long context tasks
As context windows grow, LLM performance degrades in unexpected ways.
Why it mattersIf you're hitting degraded quality on long-document tasks, this shows a planner/worker/manager decomposition that lets cheaper open models like Llama-3-70B and Qwen-72B beat single-shot GPT-4o.
articleTogether AI
Mamba-3
Meet Mamba-3: the SSM built for inference. Faster than Transformers at decode, stronger than Mamba-2, and open-source from day one.
Why it mattersMamba-3 offers a state-space alternative to Transformers with faster decode-time inference and open weights, making it worth evaluating for latency-sensitive or long-context serving where attention-based decoding is the bottleneck.
articleTogether AI
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
As GPU throughput outpaces memory bandwidth, kernels must evolve.
Why it mattersFlashAttention is the backbone of efficient transformer inference and training.
articleTogether AI
Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM serving
Serving long prompts doesn't have to mean slow responses.
Why it mattersIf you're serving long-context LLMs and fighting slow time-to-first-token, CPD shows how separating cache-warm and cache-cold workloads across prefill and decode stages can lift throughput ~40%.
articleTogether AI
How speech models fail where it matters the most and what to do about it
State-of-the-art speech models like Whisper and Deepgram score near-human on benchmarks — then fail 39% of the time on street names.
Why it mattersIf you're building on Whisper or Deepgram, near-human aggregate benchmark scores can hide catastrophic failures on critical entities like street names and proper nouns.
articleTogether AI
Consistency diffusion language models: Up to 14x faster inference without sacrificing quality
Standard diffusion language models can't use KV caching and need too many refinement steps to be practical.
Why it mattersIf you're deploying diffusion language models, CDLM removes their two biggest practical blockers — no KV caching and excessive refinement steps.
articleTogether AI
What do LLMs think when you don't tell them what to think about?
What do language models generate when you don't tell them what to generate?
Why it mattersUnderstanding that GPT defaults to code/math, Llama to narratives, DeepSeek to religious content, and Qwen to exam questions helps engineers anticipate model biases when prompts are underspecified.
articleTogether AI
articleQwen Team
Qwen3Guard: Real-time Safety for Your Token Stream
Qwen3Guard is the family's first safety guardrail model, fine-tuned from Qwen3 to classify both prompts and responses with risk levels and categories for moderation.
Why it mattersQwen3Guard is an open-weight guardrail model that classifies both prompts and responses in real time at the token-stream level, with graded risk levels and category labels across English, Chinese, and multilingual inputs.
articleQwen Team
Qwen-Image-Edit: Image Editing with Higher Quality and Efficiency
Qwen-Image-Edit extends the 20B Qwen-Image model to editing, carrying its text-rendering strength into precise text edits and feeding the input image through Qwen2.5-VL for visual semantic control.
Why it mattersQwen-Image-Edit combines semantic and appearance editing by routing the input through both a vision-language model and a VAE encoder, and it can edit rendered text precisely.
articleQwen Team
Qwen-Image: Crafting with Native Text Rendering
Qwen-Image is a 20B MMDiT image foundation model built for complex text rendering, including multi-line and paragraph-level layouts, alongside precise image editing.
Why it mattersQwen-Image is a 20B open-weights image model that finally handles complex in-image text rendering — multi-line layouts and both alphabetic and logographic scripts.
articleQwen Team
GSPO: Towards Scalable Reinforcement Learning for Language Models
Scaling RL for language models requires stable training dynamics, and existing algorithms such as GRPO can collapse irreversibly over long runs.
Why it mattersGSPO stabilizes long-horizon RL training for language models by optimizing at the sequence level rather than per-token, directly addressing the model-collapse and instability failures teams hit when scaling GRPO.
articleQwen Team
Qwen3-Coder: Agentic Coding in the World
Qwen3-Coder is Qwen's most agentic code model, led by a 480B mixture-of-experts variant with 35B active parameters, 256K native context and up to 1M tokens with extrapolation.
Why it mattersQwen3-Coder is an open-weight MoE model tuned specifically for agentic coding, tool-use, and browser-use, with 256K native (up to 1M) context and benchmark results rivaling Claude Sonnet 4.
articleQwen Team
Qwen VLo: From "Understanding" the World to "Depicting" It
Qwen VLo moves the family from understanding images to generating them, unifying multimodal comprehension and generation in one model rather than treating them as separate systems.
Why it mattersQwen VLo unifies image understanding and generation in a single model, so you can prompt it to both interpret visual content and produce high-quality recreations without stitching together separate vision and image-gen models.
articleQwen Team
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
The Qwen3 Embedding series targets text embedding, retrieval and reranking, built on the Qwen3 foundation models and leaning on their multilingual understanding for state-of-the-art results across benchmarks.
Why it mattersOpen-weight (Apache 2.0) multilingual embedding and reranking models built on Qwen3 give you a self-hostable alternative to proprietary embedding APIs for RAG and retrieval pipelines.
articleQwen Team
Qwen3: Think Deeper, Act Faster
Qwen3 arrives with the flagship Qwen3-235B-A22B competitive with DeepSeek-R1, o1, o3-mini, Grok-3 and Gemini-2.5-Pro on coding, maths and general benchmarks, alongside smaller mixture-of-experts variants.
Why it mattersQwen3 introduces hybrid reasoning (switchable thinking/non-thinking modes) and open-weight MoE models where a 30B-A3B activates only 3B parameters yet outperforms much larger models.
articleQwen Team
QVQ-Max: Think with Evidence
QVQ-Max is the first full release of Qwen's visual reasoning model, following the QVQ-72B preview, able to analyse and reason over images and video rather than only describing them.
Why it mattersQVQ-Max is a visual reasoning model that doesn't just caption images but analyzes and reasons over image and video content to solve math, programming, and creative tasks.
articleQwen Team
Qwen2.5 Omni: See, Hear, Talk, Write, Do It All!
Qwen2.5-Omni is an end-to-end multimodal model that takes text, images, audio and video and responds in real time through both text and natural speech synthesis.
Why it mattersQwen2.5-Omni is an openly available 7B end-to-end multimodal model that perceives text, images, audio, and video while streaming both text and natural speech responses in real time.
articleQwen Team
Qwen2.5-VL-32B: Smarter and Lighter
Qwen2.5-VL-32B-Instruct arrives under Apache 2.0, a reinforcement-learning-refined vision-language model at the community's favorite 32B scale.
Why it mattersQwen2.5-VL-32B-Instruct is an Apache 2.0 vision-language model at a self-hostable 32B scale, refined with reinforcement learning for stronger multimodal reasoning.
articleQwen Team
QwQ-32B: Embracing the Power of Reinforcement Learning
QwQ-32B applies scaled reinforcement learning to reasoning, following results showing RL can lift reasoning beyond what conventional pretraining and post-training reach.
Why it mattersQwQ-32B delivers reasoning performance competitive with far larger models by scaling reinforcement learning, making strong deep-thinking capability feasible to run and fine-tune at a 32B footprint.
articleQwen Team
Qwen2.5-Max: Exploring the Intelligence of Large-scale MoE Model
Qwen2.5-Max is Qwen's exploration of scaling a very large mixture-of-experts model, in a space where effective scaling experience was scarce until DeepSeek V3 disclosed its details.
Why it mattersQwen2.5-Max is a large-scale Mixture-of-Experts frontier model positioned against DeepSeek V3, giving engineers another competitive open-ecosystem option to benchmark and route to for reasoning and coding workloads.
articleQwen Team
Qwen2.5-1M: Deploy Your Own Qwen with Context Length up to 1M Tokens
Qwen2.5-1M open-sources 7B and 14B instruct checkpoints supporting a one-million-token context, together with the inference framework needed to actually serve them.
Why it mattersQwen2.5-1M lets you run open-weight 7B/14B instruct models with up to 1M-token context locally, with an included inference framework tuned for long-context throughput.
articleQwen Team
Qwen2.5 VL! Qwen2.5 VL! Qwen2.5 VL!
Qwen2.5-VL is the flagship vision-language release of the Qwen2.5 family, open in 3B, 7B and 72B base and instruct sizes, a significant step over Qwen2-VL.
Why it mattersQwen2.5-VL is an open-weight vision-language model available in 3B/7B/72B sizes, giving builders self-hostable multimodal capability (document parsing, visual grounding, long-video understanding) as an alternative to closed VLM APIs.
articleQwen Team
Global-batch load balance almost free lunch to improve your MoE LLM training
Qwen on global-batch load balancing for mixture-of-experts training, a near-free-lunch change to how router load is balanced across the global batch rather than per device.
Why it mattersIf you're training or fine-tuning MoE models, this shows how to apply load balancing at the global-batch level to keep expert utilization even.
articleQwen Team
Towards Effective Process Supervision in Mathematical Reasoning
Models reach correct answers in maths while inventing plausible reasoning steps along the way, so final-answer supervision is not enough.
Why it mattersIf you're building agents or apps that depend on multi-step reasoning, step-level process reward models catch flawed intermediate calculations that final-answer checks miss.
articleQwen Team
QVQ: To See the World with Wisdom
QVQ extends Qwen's reasoning work to vision, on the argument that human reasoning is rooted in both linguistic thought and visual memory rather than language alone.
Why it mattersQVQ extends chain-of-thought reasoning into the visual domain, letting a model reason step-by-step over images rather than just captioning them.
articleQwen Team
QwQ: Reflect Deeply on the Boundaries of the Unknown
QwQ, Qwen with Questions, is a reasoning model that approaches maths, code and general knowledge by working through uncertainty rather than answering directly.
Why it mattersQwQ is an openly available reasoning model that surfaces its self-questioning chain-of-thought, giving engineers a locally-runnable alternative to closed reasoning models for math, code, and analytical tasks.
articleQwen Team
Extending the Context Length to 1M Tokens!
Qwen2.5-Turbo extends context to one million tokens, following community demand after Qwen2.5.
Why it mattersQwen2.5-Turbo pushes usable context to ~1M tokens (roughly a million English words), enabling whole-codebase or multi-document reasoning in a single call without chunking or RAG workarounds.
articleQwen Team
Qwen2.5-Coder Series: Powerful, Diverse, Practical.
The Qwen2.5-Coder series opens as powerful, diverse and practical, with the 32B instruct variant matching GPT-4o's coding ability as the strongest open code model at release.
Why it mattersQwen2.5-Coder-32B-Instruct is a SOTA open-weight code model that rivaled GPT-4o coding performance, with a diverse size range (0.5B to 32B) letting you run local coding assistants sized to your hardware.
articleQwen Team
Qwen2.5: A Party of Foundation Models!
Qwen2.5 arrives as what the team calls possibly the largest open-source release in history, a family of foundation models built on three months of developer feedback since Qwen2.
Why it mattersQwen2.5 is one of the largest open-weight model releases available, spanning many parameter sizes with strong coding and reasoning gains — useful when you need capable, self-hostable alternatives to closed frontier APIs.
articleQwen Team
Qwen2.5-LLM: Extending the boundary of LLMs
Qwen details the Qwen2.5 language model series.
Why it mattersQwen2.5 gives you a full ladder of open-weight models (0.5B to 72B) with sizes deliberately tuned for production (10-30B) and mobile (3B) deployment.
articleQwen Team
Qwen2.5-Coder: Code More, Learn More!
Qwen2.5-Coder is the next generation of Qwen's open code models, renaming CodeQwen to Qwen-Coder and building on the CodeQwen1.5 release from earlier that year.
Why it mattersQwen2.5-Coder is a strong open-weight coding model family that can power self-hosted coding agents and IDE tooling without relying on closed APIs, giving engineers a competitive local alternative to GPT/Claude for code generation.
An index of the vibe-coding frontier. Corrections welcome.