Intel
Page 34
AlphaSignal@AlphaSignalAI
AlphaSignal ran Thinking Machines' Inkling through its private coding-agent bench: 89% resolve at $0.104 per fix, sixth of eight models, clean on easy and security tiers but dropping to 20% on expert-tier deep investigation with the field's highest turn count.
Why it mattersIndependent benchmark data on Thinking Machines' Inkling across a 13-bug repair harness — a reality check on frontier coding-agent claims.
ClaudeDevs@ClaudeDevs
Claude Code adds a screen reader mode: a flag swaps the visual terminal UI for plain linear text that VoiceOver and NVDA can follow.
Why it mattersIf you or a teammate relies on a screen reader, Claude Code is now usable via a single flag — a genuine accessibility unlock for terminal-based agentic coding.
ClaudeDevs@ClaudeDevs
Claude Code's screen reader mode adds labeled lines for you, Claude and tools, numbered menus answered with a digit, and a bell when input is needed, persistable via settings or an environment variable.
Why it mattersYou get the exact settings to enable Claude Code's accessibility mode — relevant if you or teammates rely on screen readers or want bell-on-attention terminal behavior.
Kimi K3: The open-weights escalation
An analytical article examining the release of Moonshot AI's Kimi K3, a 2.8T parameter open-weights MoE model, and its implications for the global AI ecosystem.
Why it mattersWhat Kimi K3's release signals about the open-weights race and the capability gap practitioners should plan around.
articleNathan Lambert
omny.fm
The Creator of Claude Code on The Hottest Piece of Software in the World
podcast
Listen
articleSimon Willison
Bun in Rust (Claude Code)
An analysis showing that Claude Code v2.1.181 and later ship a preview of Bun rewritten in Rust (reported as Bun v1.4.0).
Why it mattersYou now know your Claude Code installs run on pre-release Rust Bun, and you have a repeatable recipe for inspecting what runtime any agent binary actually embeds.
Peter Steinberger 🦞@steipete
Peter Steinberger watches Codex drive a browser to upload an image to a GitHub PR comment by hand, because there is no API for it, and runs his agents in VMs so they stop stealing app focus.
Why it mattersIf you run computer-use agents locally, the VM-isolation tip prevents them from hijacking your desktop, and the GitHub-upload anecdote shows how agents route around missing APIs.
ClaudeDevs@ClaudeDevs
Claude Managed Agents supports outcome-driven loops: hand Claude a rubric for task completion and it keeps working until the rubric is satisfied, enabling long-horizon autonomous work with self-correction.
Why it mattersRubric-driven looping is a practical pattern for making long-running agents self-verifying, and this points to a managed primitive that implements it for you.
Peter Steinberger 🦞@steipete
Peter Steinberger finds 5.6 Terra high underrated after switching his GitHub review bot to it: about 40% faster with negligible quality loss, better than 5.5 across the board, and far cheaper.
Why it mattersIf you run an LLM-powered code review or CI bot, this is direct guidance on a cheaper, faster model tier — and a warning that maxing reasoning effort may buy you nothing.
Alex Cheema@alexocheema
Alex Cheema sketches running Kimi K3 locally: four 512GB M3 Ultra Mac Studios, with an expected 30-plus tokens per second using MTP and tensor parallelism over Thunderbolt 5 RDMA.
Why it mattersIf you are sizing hardware for local frontier-scale MoE models, this gives ballpark memory, bandwidth, and throughput figures plus the interconnect technique (RDMA over TB5) that makes multi-Mac inference viable.
The Pelican Benchmark (SVG Pelican on a Bicycle)
An informal LLM evaluation method where models are prompted to 'Generate an SVG of a pelican riding a bicycle.' While not a rigorous benchmark, it serves as a quick 'hello world' for trying new models and gauging their cost, reasoning behavior.
Why it mattersThe pelican-on-a-bicycle SVG test — the informal benchmark that reveals model differences the official leaderboards miss.
articleSimon Willison
articleSimon Willison
xai-org/grok-build, now open source
xAI open-sources grok-build after backlash over its CLI uploading entire directories to xAI's cloud buckets.
Why it mattersA cautionary tale about what a coding agent can exfiltrate by default — worth reading before you run any new CLI agent in a sensitive directory, and the open-sourced harness is now inspectable.
articleSimon Willison
How I tricked Claude into leaking your deepest, darkest secrets
Claude's web_fetch tool is deliberately designed against data exfiltration.
Why it mattersIf your agent fetches web content and holds private data, this shows how deterministic URL restrictions fail via attacker-controlled nested links — check your own fetch tools for the same hole.
articleSimon Willison
Quoting Armin Ronacher
Armin Ronacher on the shared language of a software project being neither English nor Python.
Why it mattersIf agents let anyone change any layer without talking to its owners, you need a deliberate replacement for the synchronization that slow code review used to provide.
DOOMQL
DOOMQL asks what happens if SQLite is the game engine rather than the save file.
Why it mattersShows how far frontier models can push an absurd constraint (SQL-only game engine) and demonstrates a copy-paste prompt workflow for building live data viewers on top of a running app's database.
articleSimon Willison
Do Automated Evals Work?
We compared 100 human annotated traces against automated eval systems. Here's what we found.
Why it mattersEvidence on when LLM-judged evals agree with human annotation — and when they quietly diverge.
articleHamel Husain
Boris Cherny@bcherny
Claude Code adds /checkup, a housekeeping command that removes unused skills and MCPs, dedups and restructures CLAUDE.md files, turns off slow hooks and updates the CLI.
Why it mattersClaude Code's /checkup automates hygiene most teams do by hand: pruning unused skills, MCPs and plugins and deduping CLAUDE.md files.
AINews (Latent Space)
A daily AI newsletter that aggregates and summarizes the most significant developments across AI Twitter, Reddit, and Discord.
Why it mattersThe densest daily digest of model releases and agentic-engineering developments — effectively the industry's changelog.
articleLatent Space
cat@_catwu
a retrospective on making claude code from our early team! https://t.co/Kk9mz4BYXx
Why it mattersA first-hand retrospective from Claude Code's early team on the design calls that shaped the agent — useful pattern-matching for anyone building coding agents.
GLM-5.2
An analytical blog post about GLM-5.2, an open-weight AI model from Z.ai.
Why it mattersWhere open-weight models actually stand against the closed frontier on agentic coding, argued from GLM-5.2's benchmark results.
articleNathan Lambert
articleEugene Yan
Patterns for Building Cybersecurity Evals
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
Why it mattersPatterns for building security evals — sandboxed targets, difficulty inputs, tools and graders — transferable to any agent-eval harness.
Claude Fable 5 Analysis (Interconnects)
An opinion article analyzing Anthropic's fictional 'Claude Fable 5' model release and its safety measures.
Why it mattersIf you build on frontier models, you need to know when the vendor silently alters model behavior for certain query classes — this piece documents exactly that and how Anthropic responded.
articleNathan Lambert
The Pulse: a trend of trying to cut back on AI spend within eng departments?
The Pragmatic Engineer's Pulse on engineering departments starting to question AI ROI.
Why it mattersEngineering leaders are starting to cap per-engineer AI budgets — early signal on the ROI scrutiny coming for agent spend.
articleGergely Orosz
articleEugene Yan
Using LLMs to Secure Source Code
Build a threat model, discover vulnerabilities, verify, triage, and patch.
Why it mattersUsing LLMs across the code-security loop: threat modeling, vulnerability discovery, triage and patching.
Odd Lots
Understanding the Most Viral Chart in Artificial Intelligence
podcast
Listen
My bets on open models, mid-2026
An analytical blog post from Interconnects AI examining the capability gap between open-weight and closed AI models, forecasting economic and geopolitical dynamics shaping the open-source LLM ecosystem through 2026-2027.
Why it mattersConcrete predictions on the open-model landscape from one of the field's closest trackers — useful input for model-strategy planning.
articleNathan Lambert
The Inevitable Need for an Open Model Consortium
An opinion essay arguing that economic pressures will eventually force AI companies to form a funding consortium to sustain near-frontier open-weight models, as individual labs increasingly abandon fully-open releases in favor of profitable closed products.
Why it mattersIf your stack depends on open-weight models, this maps which sources of near-frontier open models are likely to survive and where the ecosystem is heading.
articleNathan Lambert
The Revenge of the Data Scientist
An argument that the data scientist's unusual mix of statistics and engineering, once called the sexiest job of the century and then written off, is exactly what the current moment rewards.
Why it mattersReframes the skills that matter for shipping reliable AI systems — eval design, metric selection, and debugging non-deterministic behavior.
articleHamel Husain
Lex Fridman
#494 – Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution
podcast
Listen
Evals Skills for Coding Agents
A set of skills for AI product evals, built to guard against the mistakes seen across 50-plus companies and 4,000-plus students, now that coding agents instrument applications and run the experiments themselves.
Why it mattersIf your coding agent is instrumenting and evaluating your AI product, these skills encode the error-analysis discipline that keeps it from lumping distinct failure modes into one useless score.
articleHamel Husain
A Dream of Spring for Open-Weight LLMs: 10 Architectures from Jan-Feb 2026
A technical roundup comparing ten open-weight LLM architectures released in early 2026, including Arcee Trinity, Kimi K2.5, Step 3.5 Flash, Qwen3-Coder-Next, and GLM-5.
Why it mattersA technical walkthrough of ten open-weight LLM architectures — what actually changed in MoE routing, attention and multi-token prediction.
articleSebastian Raschka, PhD
Why I Stopped Using nbdev
Hamel Husain on why he stopped using nbdev, and the more general point that a tool feels like a secret weapon only while it matches how you happen to think.
Why it mattersA concrete case study of how AI coding tools invert old tooling tradeoffs — worth reading before committing to any unconventional development workflow.
articleHamel Husain
Dwarkesh Patel
Thoughts on AI progress (Dec 2025)
podcast
Listen
Dwarkesh Patel
Satya Nadella — How Microsoft is preparing for AGI
podcast
Listen
Selecting The Right AI Evals Tool
Why the question of the best evals tool resists a direct answer.
Why it mattersIf you're choosing an evals tool for an AI product team, this walks through how expert data scientists actually assess Langsmith, Braintrust.
articleHamel Husain
articleEugene Yan
Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
Why it mattersHow to evaluate long-context Q&A systems: datasets, methodology and the benchmark pitfalls that make scores lie.
LLM Evals: Everything You Need to Know
Curated answers to the questions that came up most while teaching AI evals to 700-plus engineers and product managers, offered as sharp working opinions rather than universal rules.
Why it mattersA dense, opinionated field guide to LLM evals covering error analysis, LLM-as-judge design, and critique shadowing — the practical workflows for actually improving AI products, not just theory.
articleHamel Husain
A Field Guide to Rapidly Improving AI Products
Most AI teams invest in architecture before measurement. A field guide to improving AI products that starts by asking how you would know whether any of it works.
Why it mattersIf your AI team obsesses over frameworks and vector DBs but can't tell whether changes actually help, this lays out a measurement-first workflow.
articleHamel Husain
Using LLM-as-a-Judge For Evaluation: A Complete Guide
A practical guide to using a model as a judge, drawn from setting up evaluation systems at more than 30 companies, and the mistakes teams repeat when they try it.
Why it mattersA step-by-step methodology for building trustworthy LLM-as-a-judge systems, replacing arbitrary 1-5 scoring with 'Critique Shadowing' that anchors evals to a single domain expert's judgment.
articleHamel Husain
An Open Course on LLMs, Led by Practitioners
Mastering LLMs, an open course of workshops and talks from 25-plus practitioners covering evals, retrieval-augmented generation and fine-tuning.
Why it mattersA free, well-organized 40+ hour course distilled from a popular paid program, with annotated talks and notes from practitioners across evals, RAG, and fine-tuning — a fast way to level up on shipping real LLM products rather than toy demos.
articleHamel Husain
Netflix PRS 2024 - Applying LLMs to Recommendation Experiences
Challenges and lessons from deploying LLM experiences: evals, scalability, guardrails.
Why it mattersA real-world postmortem from Netflix on shipping LLMs into recommendation experiences at scale, with practical guidance on building evals, handling scalability, and designing guardrails that most tutorials skip.
articleEugene Yan
articleEugene Yan
Prompting Fundamentals and How to Apply them Effectively
Structured input/output, prefilling, n-shots prompting, chain-of-thought, reducing hallucinations, etc.
Why it mattersThe prompting techniques that survive contact with production: structured I/O, prefilling, n-shot selection and chain-of-thought, with when-to-use guidance.
Debugging AI With Adversarial Validation
Adversarial validation as a simple way to detect drift.
Why it mattersA cheap, infrastructure-free way to catch the silent killer of eval validity — your eval set no longer matching what production actually sends your model.
articleHamel Husain
Task-Specific LLM Evals that Do & Don't Work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
Why it mattersIt breaks down which eval approaches actually work for specific tasks like summarization, translation, and toxicity detection.
articleEugene Yan
Your AI Product Needs Evals
Hamel Husain's argument that unsuccessful LLM products almost always share one root cause, the absence of a robust evaluation system, drawn from five years of building and consulting on them.
Why it mattersThis is the reference playbook for eval systems — the single highest-leverage practice separating LLM products that improve past the demo stage from ones that stall.
articleHamel Husain
articleHamel Husain
Is Fine-Tuning Still Valuable?
Hamel Husain's answer to growing disillusionment with fine-tuning: it remains valuable in specific situations, and the case against it usually generalises from the wrong ones.
Why it mattersGives a clear decision framework for whether to invest in fine-tuning.
Evaluation & Hallucination Detection for Abstractive Summaries
Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.
Why it mattersIf you're shipping summarization features, this lays out concrete metrics — reference-based, context-based, preference-based, and self-consistency checks.
articleEugene Yan
Patterns for Building LLM-based Systems & Products
Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback.
Why it mattersA comprehensive field guide to the seven practical patterns behind production LLM systems—covering how to structure evals, RAG, caching, and guardrails—so you can build reliable AI products instead of stitching together demos.
articleEugene Yan
Some Intuition on Attention and the Transformer
What's the big deal, intuition on query-key-value vectors, multiple heads, multiple layers, and more.
Why it mattersA clear, intuition-first walkthrough of query-key-value attention, multi-head, and multi-layer mechanics — useful if you want to actually understand why the Transformer works rather than just memorize the diagram.
articleEugene Yan
More Design Patterns For Machine Learning Systems
9 patterns including HITL, hard mining, reframing, cascade, data flywheel, business rules layer, and more.
Why it mattersA compact catalog of production ML design patterns — like cascade models, hard negative mining, data flywheels, and business-rules layers.
articleEugene Yan
An index of the vibe-coding frontier. Corrections welcome.