Intel
Page 20
Pushing software engineering limits with “napkin math”
Gergely Orosz on napkin math, the practice of reasoning about system limits from a handful of memorised latency and throughput numbers, following an interview with turbopuffer's Simon Eskildsen.
Why it mattersLearn to use 'napkin math' — rough first-principles estimates of compute, I/O, and memory limits — to quickly spot why AI-era systems run slow or cost too much, illustrated through the origins of vector search infra like turbopuffer.
articleGergely Orosz
Nativ: Run AI models locally on your Mac
Nativ wraps MLX in a full macOS desktop app, giving local models a chat interface and a localhost API server, from the developer behind the MLX-VLM library.
Why it mattersNativ gives Mac users a native MLX-backed desktop app for running local LLMs and VLMs, with a chat UI plus an OpenAI-compatible localhost API server.
articleSimon Willison
A Fireside Chat with Cat and Thariq from the Claude Code team
An edited transcript of a fireside chat with Cat Wu and Thariq Shihipar of Anthropic's Claude Code team.
Why it mattersPractical prompting guidance from the people who ship Claude Code — that adding examples and long 'don't do X' lists now hurt results on frontier models, and that their system prompt shrank 80%.
articleSimon Willison
The Pelican Benchmark (SVG Pelican on a Bicycle)
An informal LLM evaluation method where models are prompted to 'Generate an SVG of a pelican riding a bicycle.' While not a rigorous benchmark, it serves as a quick 'hello world' for trying new models and gauging their cost, reasoning behavior.
Why it mattersThe pelican-on-a-bicycle SVG test — the informal benchmark that reveals model differences the official leaderboards miss.
articleSimon Willison
What does 99.9% uptime mean for inference?
Reliability numbers are easy to publish.
Why it mattersIf you depend on an inference provider, this breaks down what each uptime tier really guarantees and the failure domains it must survive, giving you concrete questions to ask before committing to an SLA.
articleTogether AI
zed v1.11.3
Zed v1.11.3 adds dedicated staged and unstaged changes multibuffers for reviewing and staging individual hunks, Git Graph improvements with Markdown commit messages.
Why it mattersZed's ACP integration and hunk-level Git review make it a fast, native editor for developers who want an agent panel and granular version control in one Rust-based tool.
repozed-zippy[bot]
Do Automated Evals Work?
We compared 100 human annotated traces against automated eval systems. Here's what we found.
Why it mattersEvidence on when LLM-judged evals agree with human annotation — and when they quietly diverge.
articleHamel Husain
ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)
ParallelKernelBench tests whether LLMs can write fast multi-GPU CUDA kernels across 87 real workloads.
Why it mattersIf you're evaluating whether to trust LLMs to generate performance-critical GPU code, ParallelKernelBench gives you a grounded reality check.
articleTogether AI
articleEugene Yan
Patterns for Building Cybersecurity Evals
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
Why it mattersPatterns for building security evals — sandboxed targets, difficulty inputs, tools and graders — transferable to any agent-eval harness.
Dwarkesh Patel
The data black hole at the center of AI
podcast
Listen
reporhys-firecrawl
firecrawl v2.11.0 — Firecrawl v2.11.0
Firecrawl v2.11.0 adds a research index over 3M+ arXiv papers and the GitHub code behind them, refreshed daily.
Why it mattersFirecrawl's v2.11 adds an arXiv+GitHub research index with SOTA arXivQA recall, keyless MCP/CLI/SDK access.
Kimi K2.7 Code vs Claude Fable 5: Landing pages that cost 94% less
We generated 12 landing pages with Kimi K2.7 Code and Claude Fable 5. Kimi cost 94% less and scored within a few points on every page. Here's what actually moved the needle.
Why it mattersIf you're picking a model for AI-generated landing pages or frontend work, this head-to-head shows Kimi K2.7 Code landing within a few points of Claude Fable 5 quality at ~94% lower cost.
articleTogether AI
articleTogether AI
Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets
How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway.
Why it mattersIf you're serving long-context or multimodal LLMs at scale, this breaks down concrete serving optimizations — KV-block-major sparse attention, paged MSA decode, and a Rust multimodal gateway.
Interconnects: Open and closed models are on different exponentials
An analysis essay from the Interconnects AI newsletter arguing that closed frontier labs (OpenAI, Anthropic) and open model builders are on divergent economic trajectories.
Why it mattersThe case that open and closed models sit on different capability exponentials — and what that means for betting on either.
articleNathan Lambert
How Together AI built the world’s fastest speech-to-text stack
Together AI built the fastest speech-to-text stack on Artificial Analysis by treating ASR as a full-path systems problem, not just a GPU inference problem.
Why it mattersIf you're serving speech-to-text at scale, this breaks down how treating ASR as an end-to-end systems problem — not just GPU kernel tuning — produces state-of-the-art latency and throughput on Artificial Analysis benchmarks.
articleTogether AI
The Pulse: a trend of trying to cut back on AI spend within eng departments?
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.
Why it mattersEngineering leaders are starting to cap per-engineer AI budgets — early signal on the ROI scrutiny coming for agent spend.
articleGergely Orosz
articleEugene Yan
Using LLMs to Secure Source Code
Build a threat model, discover vulnerabilities, verify, triage, and patch.
Why it mattersUsing LLMs across the code-security loop: threat modeling, vulnerability discovery, triage and patching.
Serving DeepSeek-V4: why million-token context is an inference systems problem
DeepSeek-V4 makes million-token context a serving-systems problem.
Why it mattersIf you're serving million-token-context workloads, this breaks down the real bottlenecks — compressed KV cache layouts, prefix caching, and kernel maturity on HGX B200.
articleTogether AI
Odd Lots
Understanding the Most Viral Chart in Artificial Intelligence
podcast
Listen
Accelerate RL rollouts by up to 50% with distribution-aware speculative decoding
Rollout is the silent bottleneck in RL post-training. DAS fixes it with adaptive speculative decoding — up to 50% faster, zero degradation in reward quality.
Why it mattersIf you're doing RL post-training, rollout generation dominates wall-clock time.
articleTogether AI
My bets on open models, mid-2026
An analytical blog post from Interconnects AI examining the capability gap between open-weight and closed AI models, forecasting economic and geopolitical dynamics shaping the open-source LLM ecosystem through 2026-2027.
Why it mattersConcrete predictions on the open-model landscape from one of the field's closest trackers — useful input for model-strategy planning.
articleNathan Lambert
Parcae: Doing more with fewer parameters using stable looped models
Parcae is a stable looped language model that matches the quality of a Transformer twice its size — a 770M model reaching 1.3B-level performance.
Why it mattersParcae shows that adding recurrence (looping a smaller model) can match a Transformer twice its size, offering a compute- and memory-efficient path to quality — useful if you're weighing model size against inference cost.
articleTogether AI
EinsteinArena: Harnessing the collective intelligence of agents in the wild to advance science
EinsteinArena is a platform where AI agents collaborate and compete on open math problems.
Why it mattersEinsteinArena demonstrates that competitive/collaborative multi-agent setups can push the frontier on genuinely open math problems, not just closed benchmarks.
articleTogether AI
AI for Systems: Using LLMs to Optimize Database Query Execution
New research shows LLMs can optimize database query execution plans—achieving up to 4.78x speedups by correcting the cardinality estimation errors that statistical heuristics miss.
Why it mattersIf you build data-intensive systems, this shows LLMs can outperform traditional statistical heuristics on cardinality estimation—the classic weak point in query planners.
articleTogether AI
Inside the Together AI kernels team
The team behind FlashAttention and ThunderKittens — how Together AI's kernel researchers close the gap between GPU hardware and production AI.
Why it mattersFlashAttention and ThunderKittens are foundational to modern LLM inference and training performance, so understanding how this team writes GPU kernels gives engineers insight into where real throughput gains come from.
articleTogether AI
Aurora
1.25x over a well-trained static speculator.
Why it mattersIf you're serving LLMs and using speculative decoding, Aurora replaces one-time offline draft-model training with an RL loop that adapts the speculator to your live request distribution, reportedly yielding 1.25x throughput over a well-tuned static speculator.
articleTogether AI
Plan, divide, and conquer: How weak models excel at long context tasks
As context windows grow, LLM performance degrades in unexpected ways.
Why it mattersIf you're hitting degraded quality on long-document tasks, this shows a planner/worker/manager decomposition that lets cheaper open models like Llama-3-70B and Qwen-72B beat single-shot GPT-4o.
articleTogether AI
Lex Fridman
#494 – Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution
podcast
Listen
Mamba-3
Meet Mamba-3: the SSM built for inference. Faster than Transformers at decode, stronger than Mamba-2, and open-source from day one.
Why it mattersMamba-3 offers a state-space alternative to Transformers with faster decode-time inference and open weights, making it worth evaluating for latency-sensitive or long-context serving where attention-based decoding is the bottleneck.
articleTogether AI
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
As GPU throughput outpaces memory bandwidth, kernels must evolve.
Why it mattersFlashAttention is the backbone of efficient transformer inference and training.
articleTogether AI
Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM serving
Serving long prompts doesn't have to mean slow responses.
Why it mattersIf you're serving long-context LLMs and fighting slow time-to-first-token, CPD shows how separating cache-warm and cache-cold workloads across prefill and decode stages can lift throughput ~40%.
articleTogether AI
How speech models fail where it matters the most and what to do about it
State-of-the-art speech models like Whisper and Deepgram score near-human on benchmarks — then fail 39% of the time on street names.
Why it mattersIf you're building on Whisper or Deepgram, near-human aggregate benchmark scores can hide catastrophic failures on critical entities like street names and proper nouns.
articleTogether AI
Consistency diffusion language models: Up to 14x faster inference without sacrificing quality
Standard diffusion language models can't use KV caching and need too many refinement steps to be practical.
Why it mattersIf you're deploying diffusion language models, CDLM removes their two biggest practical blockers — no KV caching and excessive refinement steps.
articleTogether AI
What do LLMs think when you don't tell them what to think about?
What do language models generate when you don't tell them what to generate?
Why it mattersUnderstanding that GPT defaults to code/math, Llama to narratives, DeepSeek to religious content, and Qwen to exam questions helps engineers anticipate model biases when prompts are underspecified.
articleTogether AI
Dwarkesh Patel
Thoughts on AI progress (Dec 2025)
podcast
Listen
Dwarkesh Patel
Satya Nadella — How Microsoft is preparing for AGI
podcast
Listen
articleQwen Team
Qwen3Guard: Real-time Safety for Your Token Stream
Qwen3Guard is the family's first safety guardrail model, fine-tuned from Qwen3 to classify both prompts and responses with risk levels and categories for moderation.
Why it mattersQwen3Guard is an open-weight guardrail model that classifies both prompts and responses in real time at the token-stream level, with graded risk levels and category labels across English, Chinese, and multilingual inputs.
articleQwen Team
Qwen-Image-Edit: Image Editing with Higher Quality and Efficiency
Qwen-Image-Edit extends the 20B Qwen-Image model to editing, carrying its text-rendering strength into precise text edits and feeding the input image through Qwen2.5-VL for visual semantic control.
Why it mattersQwen-Image-Edit combines semantic and appearance editing by routing the input through both a vision-language model and a VAE encoder, and it can edit rendered text precisely.
articleQwen Team
Qwen-Image: Crafting with Native Text Rendering
Qwen-Image is a 20B MMDiT image foundation model built for complex text rendering, including multi-line and paragraph-level layouts, alongside precise image editing.
Why it mattersQwen-Image is a 20B open-weights image model that finally handles complex in-image text rendering — multi-line layouts and both alphabetic and logographic scripts.
articleQwen Team
GSPO: Towards Scalable Reinforcement Learning for Language Models
Scaling RL for language models requires stable training dynamics, and existing algorithms such as GRPO can collapse irreversibly over long runs.
Why it mattersGSPO stabilizes long-horizon RL training for language models by optimizing at the sequence level rather than per-token, directly addressing the model-collapse and instability failures teams hit when scaling GRPO.
articleQwen Team
Qwen3-Coder: Agentic Coding in the World
Qwen3-Coder is Qwen's most agentic code model, led by a 480B mixture-of-experts variant with 35B active parameters, 256K native context and up to 1M tokens with extrapolation.
Why it mattersQwen3-Coder is an open-weight MoE model tuned specifically for agentic coding, tool-use, and browser-use, with 256K native (up to 1M) context and benchmark results rivaling Claude Sonnet 4.
articleQwen Team
Qwen VLo: From "Understanding" the World to "Depicting" It
Qwen VLo moves the family from understanding images to generating them, unifying multimodal comprehension and generation in one model rather than treating them as separate systems.
Why it mattersQwen VLo unifies image understanding and generation in a single model, so you can prompt it to both interpret visual content and produce high-quality recreations without stitching together separate vision and image-gen models.
articleEugene Yan
Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
Why it mattersHow to evaluate long-context Q&A systems: datasets, methodology and the benchmark pitfalls that make scores lie.
articleQwen Team
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
The Qwen3 Embedding series targets text embedding, retrieval and reranking, built on the Qwen3 foundation models and leaning on their multilingual understanding for state-of-the-art results across benchmarks.
Why it mattersOpen-weight (Apache 2.0) multilingual embedding and reranking models built on Qwen3 give you a self-hostable alternative to proprietary embedding APIs for RAG and retrieval pipelines.
articleQwen Team
Qwen3: Think Deeper, Act Faster
Qwen3 arrives with the flagship Qwen3-235B-A22B competitive with DeepSeek-R1, o1, o3-mini, Grok-3 and Gemini-2.5-Pro on coding, maths and general benchmarks, alongside smaller mixture-of-experts variants.
Why it mattersQwen3 introduces hybrid reasoning (switchable thinking/non-thinking modes) and open-weight MoE models where a 30B-A3B activates only 3B parameters yet outperforms much larger models.
articleQwen Team
QVQ-Max: Think with Evidence
QVQ-Max is the first full release of Qwen's visual reasoning model, following the QVQ-72B preview, able to analyse and reason over images and video rather than only describing them.
Why it mattersQVQ-Max is a visual reasoning model that doesn't just caption images but analyzes and reasons over image and video content to solve math, programming, and creative tasks.
articleQwen Team
Qwen2.5 Omni: See, Hear, Talk, Write, Do It All!
Qwen2.5-Omni is an end-to-end multimodal model that takes text, images, audio and video and responds in real time through both text and natural speech synthesis.
Why it mattersQwen2.5-Omni is an openly available 7B end-to-end multimodal model that perceives text, images, audio, and video while streaming both text and natural speech responses in real time.
articleQwen Team
Qwen2.5-VL-32B: Smarter and Lighter
QWEN CHAT GITHUB HUGGING FACE MODELSCOPE DISCORD Introduction At the end of January this year, we launched the Qwen2.5-VL series of models.
Why it mattersQwen2.5-VL-32B-Instruct is an Apache 2.0 vision-language model at a self-hostable 32B scale, refined with reinforcement learning for stronger multimodal reasoning.
articleQwen Team
QwQ-32B: Embracing the Power of Reinforcement Learning
QwQ-32B applies scaled reinforcement learning to reasoning, following results showing RL can lift reasoning beyond what conventional pretraining and post-training reach.
Why it mattersQwQ-32B delivers reasoning performance competitive with far larger models by scaling reinforcement learning, making strong deep-thinking capability feasible to run and fine-tune at a 32B footprint.
articleEugene Yan
Prompting Fundamentals and How to Apply them Effectively
Structured input/output, prefilling, n-shots prompting, chain-of-thought, reducing hallucinations, etc.
Why it mattersThe prompting techniques that survive contact with production: structured I/O, prefilling, n-shot selection and chain-of-thought, with when-to-use guidance.
An index of the vibe-coding frontier. Corrections welcome.