Intel
Page 46
Understanding the 4 Main Approaches to LLM Evaluation (From Scratch)
An educational article that explains the four primary methods for evaluating large language models — multiple-choice benchmarks, verifiers, leaderboards, and LLM judges.
Why it mattersIt gives a hands-on, from-scratch breakdown of the four main LLM evaluation approaches — including how MMLU scoring and verifier-based evals actually work.
articleSebastian Raschka, PhD
Dwarkesh Patel
Some thoughts on the Sutton interview
podcast
Listen
Tencent Hy@TencentHunyuan
Two Tencent open-source drops landed at once: HunyuanImage 3.0, an 80B-parameter text-to-image model, and Hunyuan3D-Part, which pairs P3-SAM native 3D part segmentation with X-Part for controllable shape generation. Both are free to deploy.
Why it mattersAn 80B-parameter open text-to-image model and a part-level 3D generator with native part segmentation are both deployable without a vendor API, widening what builders can run on their own hardware.
Selecting The Right AI Evals Tool
Why the question of the best evals tool resists a direct answer.
Why it mattersIf you're choosing an evals tool for an AI product team, this walks through how expert data scientists actually assess Langsmith, Braintrust.
articleHamel Husain
Andrew Ng@AndrewYNg
LandingAI's Agentic Document Extraction moves to a Document Pre-trained Transformer aimed at documents that break conventional parsers — large, complex tables above all — and ships an SDK that reduces a call to a few lines of code.
Why it mattersDocument extraction from large, messy tables is a common failure point in finance and healthcare pipelines; DPT targets exactly that case and the new SDK cuts integration to a handful of lines.
Real AI Agents and Real Work
An analytical essay by Ethan Mollick (One Useful Thing) examining how AI agents have crossed a threshold into performing economically valuable tasks.
Why it mattersA grounded read on where AI agents actually produce economic value versus noise, anchored by a concrete demo of Claude Sonnet 4.5 reproducing an academic paper's findings.
articleEthan Mollick
Effective Context Engineering for AI Agents
Anthropic's guidance on treating an agent's context as a scarce budget rather than a bucket.
Why it mattersFrames the context window as a finite resource to be curated deliberately — what to load, when to retrieve, and what to compact — which is the main lever on agent reliability over long runs.
articlewww.anthropic.com
Updates To Our Consumer Terms
Anthropic's consumer terms now default Free, Pro, and Max accounts into model training, including Claude Code run from those accounts.
Why it mattersIf you run Claude Code on a Pro or Max account, your coding sessions are training data by default and retained for five years unless you change the setting. Commercial and API use is unaffected.
articleAnthropic News
Dwarkesh Patel
Richard Sutton – Father of RL thinks LLMs are a dead end
podcast
Listen
Andrew Ng@AndrewYNg
Reading China's ban on major tech firms buying Nvidia as a self-sufficiency signal: DeepSeek-R1-Safe trained on 1000 Ascend chips, and Huawei's CloudMatrix 384 answers the GB200 with more, weaker chips orchestrated at system level. TSMC concentration is the mirror risk.
Why it mattersChina cutting off Nvidia purchases is evidence its domestic accelerators are good enough at system level, which reshapes assumptions about where frontier training capacity — and Taiwan concentration risk — sits.
Autoregressive To Diffusion Vlms
Runway converts a pretrained autoregressive vision-language model into a diffusion decoder, using block-wise decoding and a confidence threshold to enable parallel token generation without paying diffusion pretraining costs.
Why it mattersDiffusion decoding gives VLMs a tunable speed/quality dial, and adapting an existing autoregressive model avoids the ~16x training compute that diffusion language modeling normally demands.
articleRunway editorial sitemap
articleQwen Team
Qwen3Guard: Real-time Safety for Your Token Stream
Qwen3Guard is the family's first safety guardrail model, fine-tuned from Qwen3 to classify both prompts and responses with risk levels and categories for moderation.
Why it mattersQwen3Guard is an open-weight guardrail model that classifies both prompts and responses in real time at the token-stream level, with graded risk levels and category labels across English, Chinese, and multilingual inputs.
Andrew Ng@AndrewYNg
The case for having agents write the tests around code agents write, backed by real misbehaviour: infra bugs that took humans weeks to find, a password-reset security hole added for convenience, reward hacking on test code, and an agent running rm *.py in a working directory.
Why it mattersCoding agents introduce specific, recurring failure classes — including editing tests to make them pass — and this makes the case for testing infrastructure code hardest, where downstream debugging cost is highest.
A Postmortem of Three Recent Issues
Anthropic's technical report examines three bugs that intermittently degraded Claude's response quality, detailing the root causes, why the issues took time to diagnose and fix.
Why it mattersIf you build on Claude, this explains concretely how inference-stack issues — routing, precision, and compiler-level bugs.
articleAnthropic
xAI’s Colossus 2 – First Gigawatt Datacenter In The World, Unique RL Methodology, Capital Raise
Estimates from a proprietary datacenter model on xAI's Colossus 2 gigawatt build, how its single-cluster capacity compares to Meta and Anthropic.
Why it mattersSingle-coherent cluster size bounds what frontier training runs are possible and when; this puts dated capacity numbers on who will hold the largest one.
articleJeremie Eliahou Ontiveros
Claude In Xcode
Claude reached general availability in Xcode 26, wiring Sonnet 4 into Apple's IDE for conversational code work, documentation generation, SwiftUI previews and inline edits.
Why it mattersApple-platform developers can drive Xcode's coding intelligence with their existing Claude subscription, and the usage budget is shared with other Claude Code surfaces rather than being additive.
articleAnthropic News
Training an LLM-RecSys Hybrid for Steerable Recs with Semantic IDs
An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.
Why it mattersIt shows how to fold recommendation capability directly into an LLM via semantic item IDs, eliminating separate retrieval/tool infrastructure — useful for anyone building steerable, conversational recommender systems.
articleEugene Yan
Dwarkesh Patel
Fully autonomous robots are much closer than you think – Sergey Levine
podcast
Listen
On Working with Wizards
An essay by Ethan Mollick exploring how the nature of working with AI is shifting from collaborative 'co-intelligence' to a 'wizard' model, where AI produces sophisticated outputs through opaque processes that users can only verify, not shape.
Why it mattersIt gives practitioners a sharp mental model for working with frontier models like GPT-5 Pro and Claude Opus.
articleEthan Mollick
Writing Effective Tools for AI Agents—Using AI Agents
Anthropic's engineering team on making tools legible to models rather than to humans — scoping, naming, and failure surfaces.
Why it mattersTool definitions are the interface your agent actually reasons over; this covers how to shape names, scopes and error surfaces, and how to use agents themselves to iterate on them.
articlewww.anthropic.com
Another Giant Leap: The Rubin CPX Specialized Accelerator & Rack
Nvidia's Rubin CPX puts prefill on its own compute-heavy, bandwidth-light die, making disaggregated serving a hardware property rather than a software trick.
Why it mattersSplitting prefill and decode onto different silicon changes the economics of long-context inference — the phase that dominates cost when agents feed large contexts on every turn.
articleDylan Patel
Huawei Ascend Production Ramp: Die Banks, TSMC Continued Production, HBM is The Bottleneck
A supply-side read on Huawei's Ascend ramp: accumulated die banks, TSMC-origin inventory, and high-bandwidth memory as the real bottleneck.
Why it mattersHBM, not logic, is the ceiling on China's accelerator output; that constraint shapes how much non-Western compute exists and where open-weight models can actually be trained and served.
articleDylan Patel
Understanding and Implementing Qwen3 From Scratch
An in-depth educational article that walks through the Qwen3 LLM architecture and reimplements it from scratch in pure PyTorch.
Why it mattersIf you want to actually understand how modern open-weight LLMs work, this walks you through reimplementing Qwen3's dense and Mixture-of-Experts variants in pure PyTorch.
articleSebastian Raschka, PhD
Using Linters to Direct Agents
This Factory.ai blog post argues that as AI coding agents take over more of the actual code writing, linters should be used to encode architectural rules, boundaries.
Why it mattersIf you're running coding agents at any scale, deterministic lint rules are a far more reliable steering mechanism than prompt-level instructions or CLAUDE.md conventions the agent may ignore.
articleFactory
Updating Restrictions Of Sales To Unsupported Regions
Anthropic tightened its regional restrictions so that entities more than 50% owned, directly or indirectly, by companies headquartered in unsupported jurisdictions are barred from its services wherever they operate.
Why it mattersEligibility for Anthropic's services now turns on ownership structure, not incorporation address. A majority-foreign-owned subsidiary building on Claude can be cut off even when it operates in a supported country.
articleAnthropic News
Amazon’s AI Resurgence: AWS & Anthropic’s Multi-Gigawatt Trainium Expansion
SemiAnalysis argues AWS's accelerator-cloud lag is about to reverse, with Anthropic's fivefold revenue growth anchoring a multi-gigawatt Trainium buildout.
Why it mattersWhere a frontier lab's training and inference capacity gets built determines which silicon and which cloud your model access rides on; this lays out the Trainium-Anthropic dependency in concrete capacity terms.
articleJeremie Eliahou Ontiveros
Le Chat. Custom MCP connectors. Memories.
MCP becomes the extensibility path for Mistral's assistant.
Why it mattersLe Chat ships more than 20 MCP connectors, including Databricks, Snowflake, GitHub, Atlassian, Stripe, and Zapier, and accepts custom MCP servers, with on-prem or in-cloud deployment.
articlemistral.ai
Mass Intelligence
An analytical essay by Ethan Mollick (One Useful Thing) exploring how powerful AI is becoming as accessible as a Google search, covering the economics, ease of use.
Why it mattersA clear-eyed framing of what happens when advanced AI becomes as cheap and ubiquitous as web search, with concrete cost/energy figures and capability demos that help builders reason about pricing, adoption.
articleEthan Mollick
Andrew Ng@AndrewYNg
An argument that running agents in parallel is emerging as a distinct scaling axis next to training and test-time compute — cheap tokens plus latency pressure are pushing research fetches, multi-agent coding across worktrees, and other workflows from sequential to fan-out.
Why it mattersReframes agent latency as a design choice: work that reads as sequential reasoning can often be fanned out, buying quality without making users wait as per-token cost falls.
Developing Computer Use
Anthropic's launch write-up for computer use explains how Claude was taught to read screenshots and count pixels to place a cursor, why it generalized from a deliberately small training surface, and where it still fails.
Why it mattersScreen-driven computer control was trained on a handful of simple applications and generalized, and the OSWorld number tells you how far from human reliability such agents were at launch.
articleAnthropic News
Building Production Ready Agentic Systems
Shopify details how Sidekick grew from a tool-calling assistant into a production agentic system.
Why it mattersShows how a production agent handles tool sprawl and how LLM-judge evaluation plus GRPO training were wired in — a rare look at what it takes to keep an agentic loop reliable at scale.
articleShopify Engineering
Post Training In The Model Factory
Post-training reuses almost everything the pretraining stack already has.
Why it mattersShows post-training at scale as an orchestration problem reusing pretraining infrastructure, and names the two gaps worth building: dataset inspection and direct GPU-to-GPU weight transfer between training and inference.
articlePoolside editorial sitemap
Runway Game Worlds
Runway opens a beta for generated interactive fiction.
Why it mattersA frontier video lab shipping playable, generated-as-you-play narrative worlds shows where its world-model work is being productized, with real-time video generation stated as the next step in the same interface.
articleRunway editorial sitemap
H100 vs GB200 NVL72 Training Benchmarks – Power, TCO, and Reliability Analysis, Software Improvement Over Time
Measured training benchmarks across thousands of H100s and GB200 NVL72 racks, scoring model FLOPs utilization, cost per million tokens trained, and energy draw.
Why it mattersNvidia's generational claims collapse once downtime and lost engineering time enter the math; these are measured MFU and cost-per-token figures across cluster scales and software versions you can reason about directly.
articleDylan Patel
articleQwen Team
Qwen-Image-Edit: Image Editing with Higher Quality and Efficiency
Qwen-Image-Edit extends the 20B Qwen-Image model to editing, carrying its text-rendering strength into precise text edits and feeding the input image through Qwen2.5-VL for visual semantic control.
Why it mattersQwen-Image-Edit combines semantic and appearance editing by routing the input through both a vision-language model and a VAE encoder, and it can edit rendered text precisely.
Trust Calibration for AI Software Builders
Trust calibration — matching how much users trust an AI feature to what it can actually do.
Why it mattersNames a design target most AI products miss: aligning user trust with actual model capability, since over-trust produces destructive automation and under-trust makes a capable product feel useless.
articlefly.io
Block Diffusion Interpolating Between Autoregressive And Diffusion Language Models 2025 03 12
Diffusion language models offer parallel decoding and controllability but have been stuck with fixed-length output and no KV cache.
Why it mattersDiffusion LMs promise parallel, controllable generation but choke on fixed lengths and lose KV caching. Block-wise interpolation with autoregressive modeling recovers both while keeping the parallelism.
articleCohere editorial sitemap
Back To Basics Revisiting Reinforce Style Optimization For Learning From Human Feedback In Llms 2024 02 23
RLHF pipelines inherited PPO from a very different RL setting.
Why it mattersIf you are running RLHF, the PPO scaffolding you inherited may be unneeded overhead: simpler REINFORCE-style optimization performs competitively while being cheaper and easier to tune.
articleCohere editorial sitemap
Aya Model Paper 2024 02 13
Breakthroughs have concentrated in a handful of data-rich languages.
Why it mattersAn openly released instruction-tuned model spanning far more languages than typical open checkpoints, giving a starting point for languages that mainstream open models barely cover.
articleCohere editorial sitemap
25 01 16 Finetuning
Black Forest Labs opens finetuning on its pro model line, letting teams teach it specific people, products, or styles from a handful of images.
Why it mattersTeams can customize a flagship closed image model from 1 to 5 reference images, then reuse that finetune across inpainting and depth-conditioned endpoints, achieving brand or character consistency without training a separate model per tool.
articleBlack Forest Labs editorial sitemap
24 08 01 Bfl
Black Forest Labs' founding announcement, notable chiefly for shipping the FLUX.1 suite in three tiers.
Why it mattersThe original FLUX.1 release established the pro/dev/schnell split that still shapes open-weight image tooling: a distilled fast variant, a non-commercial open checkpoint, and an API-only flagship.
articleBlack Forest Labs editorial sitemap
25 01 03 Nvidia
Black Forest Labs details FP4-optimized FLUX builds for RTX 50-series Blackwell GPUs, reporting FLUX.1 [dev] at 10GB VRAM and twice the 4090's BF16 speed.
Why it mattersFP4 quantization brings FLUX.1 [dev] down to 10GB VRAM while roughly doubling throughput on Blackwell RTX cards, changing what image generation is practical to run locally rather than via API.
articleBlack Forest Labs editorial sitemap
Aya Expanse Combining Research Breakthroughs For A New Multilingual Frontier 2024 12 06
Aya Expanse packages several of Cohere Labs' separate research lines, including teacher routing for synthetic data, multilingual preference training.
Why it mattersTwo new open-weight multilingual models at 8B and 32B, with the full post-training recipe covering synthetic data routing, preference training, and merging laid out rather than hinted at.
articleCohere editorial sitemap
The Carrier And The Beacon
Poolside's inference layer has to serve two incompatible masters: long-running agent loops that want throughput and completions that want latency.
Why it mattersAgentic workloads and code completion make opposite demands on an inference stack, throughput versus latency, and that tension, not raw speed, is what shapes a serving codebase used for both training-time evals and production.
articlePoolside editorial sitemap
Flux 1 Kontext
Black Forest Labs' Kontext suite folds text-to-image and instruction editing into a single flow-matching model, adding character consistency across scenes, style reference.
Why it mattersIn-context editing removes finetuning and mask workflows from image pipelines: you condition on a reference image, edit by instruction, and stack edits while identity holds.
articleBlack Forest Labs editorial sitemap
GPT-5 Set the Stage for Ad Monetization and the SuperApp
The argument that GPT-5's real-time router, not its benchmark scores, was the actual release.
Why it mattersThe router, not the model card, decides which model answers a request — which drives inference cost, latency variance, and how a consumer-scale free tier gets subsidized.
articleDoug OLaughlin
Guardrails for money movement: Integrating Stripe MCP with Portia AI
Stripe on building payment agents with guardrails: integrating Stripe MCP with Portia AI so money movement runs through built-in constraints and human approval workflows.
Why it mattersAgents that move money are the clearest test of whether your approval and guardrail design actually holds.
articleMounir Mouawad
Scaling the Memory Wall: The Rise and Roadmap of HBM
A walkthrough of high-bandwidth memory from stack manufacturing to the HBM4 custom base die, tied directly to serving decisions.
Why it mattersMemory bandwidth and capacity — not FLOPs — set the ceiling on KV-cache size, context length, and serving cost; this explains the roadmap that will determine those limits for the next accelerator generations.
articleDylan Patel
Designing A World Class Code Execution Environment
How Poolside makes reinforcement learning from code execution feasible.
Why it mattersA frontier lab's blueprint for turning arbitrary repositories into runnable container images at scale, including using agents to fix the ones that will not build and layering revisions so variants stay cheap.
articlePoolside editorial sitemap
Games as Model Eval: 1-Click Deploy AI Town on Fly.io
Why leaderboard benchmarks tell you little about how a model behaves over long contexts or in character, and how game environments — Kaggle Game Arena, a one-click AI Town deploy.
Why it mattersFrames games and simulated social environments as evals that surface long-context and in-character failures benchmarks miss, with a deployable AI Town instance as a way to run the comparison yourself.
articlefly.io
An index of the vibe-coding frontier. Corrections welcome.