Intel
Page 14
SemiAnalysis@SemiAnalysis_
AMD MI355X vLLM HAS BEATEN B200 vLLM ON KIMI K2.5 (THE SAME MODEL ARCH AS XAI CURSOR COMPOSER 2.5)🚀🚨 This uses upstream AMD kernels from the @GPU_MODE community. We explain below 👇 1/6🧵 https://t.co/98vEM7Z2vK
Why it mattersUpstream AMD inference beating Nvidia on a real model is the first hard evidence the CUDA gap is closing where it counts.
Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd
David Brumley, who founded picoCTF and coached pwn2own winners, argues you teach a model to hack the way you teach a person.
Why it mattersOffensive-capability evaluation has no clean ground truth, and this is a concrete curriculum-plus-measurement approach.
videoAI Engineer
Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software
Everyone wants agents that handle long-horizon work without agreeing what long-horizon means.
Why it mattersThe time-horizon numbers everyone quotes for agent capability rest on a metric that is shakier than it looks.
videoAI Engineer
articleOpenAI
Ten advances in mathematics and theoretical computer science
OpenAI shares new results on long-standing open problems in mathematics and theoretical computer science, including advances in geometry, cryptography, and complexity.
Why it mattersResults on genuinely open problems are among the clearest available signals of where frontier reasoning capability actually stands.
What's Next After RLHF? — Diogo Almeida, TypeSafe AI
RLHF made models extraordinary at pleasing the human in the loop, which a GPT-4 co-author argues is exactly the problem.
Why it mattersSycophancy and overclaiming are not bugs on top of RLHF; the argument is that they are what it optimizes for.
videoAI Engineer
Stateless MCP has recaptured my interest (and inspired mcp-explorer and datasette-mcp)
The 2026-07-28 MCP specification makes the protocol stateless, the biggest change since it launched.
Why it mattersStateless MCP changes how servers are built and deployed, and anyone maintaining an MCP integration needs to know what moved.
articleSimon Willison
Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI
Swap compute for data on the scaling curve and the same money buys a better model, which is why Ari Morcos calls data quality the compute multiplier.
Why it mattersIf curation buys the same gains as more compute, the training budget should move.
videoAI Engineer
Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute
Applied Compute trains custom models with reinforcement learning that plug into whatever harness an enterprise already runs, using an orchestrator that fans interactions to inference engines, collects graded rollouts and feeds a training engine.
Why it mattersIt shows the actual moving parts of keeping a deployed model learning from its own production traffic.
videoAI Engineer
Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs
The hard part of post-training is not the algorithm but the data and environments feeding it, and as agents run autonomously for hours something eventually falls over.
Why it mattersReliability over long autonomous runs is an environment-and-data problem before it is an algorithm problem.
videoAI Engineer
Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning
Ross Taylor worked on Galactica in 2022, an early science model that crossed the line on curated data and intermediate reasoning tokens before the backlash overshadowed it.
Why it mattersLong-horizon RL is where agent reliability is currently being won or lost.
videoAI Engineer
smevals - a small eval suite for evaluating models, prompts, and harnesses
smevals is a small eval suite for comparing models, prompts and harnesses, built with Jesse Vincent's Prime Radiant lab to answer capability questions by running matched suites across configurations and grading the results.
Why it mattersA lightweight harness for running the same eval across model and prompt configurations is the missing piece in most ad-hoc evaluation setups.
articleSimon Willison
Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang
Training an agent to run production software needs data that looks like production.
Why it mattersAgents trained on clean diffs will not be ready for the 3am database failure they are being aimed at.
videoAI Engineer
The Base Model Is Dead — Varun Singh, Arcee AI
The story that a base model mirrors the internet is dead, argues Arcee AI's pre-training lead.
Why it mattersIf base checkpoints already carry instruction and reasoning behaviour, assumptions behind fine-tuning and evaluation need revising.
videoAI Engineer
Verifiable Environments for AI in Biology — Kenny Workman, LatchBio
A single spatial biology run yields two to six terabytes, far past what a scientist can eyeball.
Why it mattersFinding domains where correctness is machine-checkable is the bottleneck in building RL environments.
videoAI Engineer
Ending AI Slop — Thais Castello Branco, Taste Labs
AI remains badly behind on subjective work, the writing and design where quality is real but hard to pin down.
Why it mattersRL environments are easy where answers verify; this is a framework for the majority of work where they do not.
videoAI Engineer
articleSimon Willison
datasette-agent 0.4a0
Release: datasette-agent 0.4a0 New await context.browser_task() mechanism allowing agent tools to run code directly in the user's browser. #33 This is an exciting new capability.
Why it mattersLetting agent tools run code in the user's own browser opens a tool-execution path that does not need server-side sandboxing.
Read more about how we set up our cloud agent environment:
Read more about how we set up our cloud agent environment: https://t.co/sVSv1HpBn9
Why it mattersHow a major coding-agent vendor isolates and provisions cloud agent sandboxes is directly reusable by teams building their own agent runtimes.
articlecursor_ai
articleOpenAI
Advancing the price-performance frontier with GPT-5.6
Explore lower GPT‑5.6 pricing for Luna and Terra—and how OpenAI’s more efficient models help enterprises deploy AI workflows at scale.
Why it mattersPrice cuts on a frontier model directly change the unit economics of running agents at scale.
Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
Repository-level completion is usually approached with semantic indexing or graph analysis, both of which cost heavily to build and maintain.
Why it mattersTeams reach for embeddings and code graphs by default; this measures whether lexical search already gets there for less.
articleBaoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin
When Knowledge Changes: Metamorphic Testing of RAG Systems with Mutations
RAG evaluation checks correctness against a static snapshot, so routine updates and factual changes go undetected.
Why it mattersYour RAG evals pass on yesterday's snapshot while a routine document update quietly breaks answers.
articleJinhan Kim, Samuele Pasini, Paolo Tonella
A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities
Open source projects have written contribution rules for coding agents, from outright bans to disclosure requirements and human sign-offs.
Why it mattersRepos are writing agent rules on the assumption agents will follow them; this checks whether they do.
articleWenhao Yang, Runzhi He, Minghui Zhou
Graph Is the Verifier: Agentic Reinforcement Learning for Interprocedural Vulnerability Detection
On a sample of real CVEs, 71.7% of vulnerable functions need evidence from outside the function to classify correctly.
Why it mattersFunction-level vulnerability detectors are structurally blind to most real CVEs; agentic traversal is a credible fix.
articleYikun Li, Ting Zhang, Jiakun Liu, Jinfeng Jiang, Yuheng Yieh, Yixin Yang, Wen Bin Leow, Yide Yin, Yintong Huo, Eng Lieh Ouh, Lwin Khin Shar, David Lo
Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms
Serial coding performance is well covered; parallel programming is not.
Why it mattersCoding agents that look competent on serial code degrade on parallel algorithms — worth knowing before delegating that work.
articleShiqi Cheng, Evelyne Ringoot, Rabab Alomairy, Alan Edelman
Fantastic Adaptive Taxonomies and How to Use Them
Raw execution traces are a poor medium for accumulating feedback: long, instance-specific, and without stable vocabulary for recurring failures.
Why it mattersEvery weight-free agent improvement loop reads traces; giving those traces a stable structure is what makes the feedback compound.
articleMert Cemri, Andrei Cojocaru, Melissa Pan, Shu Liu, Shubham Agarwal, Alexander Krentsel, Jay Tang, Kannan Ramchandran, Joseph E. Gonzalez, Matei Zaharia, Alex Dimakis, Ion Stoica
Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions
A study of 409 fixed bugs across five agentic frameworks including CrewAI and AutoGen, classifying symptoms, root causes and triggering conditions specific to autonomous multi-agent systems rather than earlier LLM pipelines.
Why it mattersKnowing how the agent framework itself fails is prerequisite to debugging anything built on top of it.
articleXiaowen Zhang, Hannuo Zhang, Shin Hwei Tan
REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
A/B testing takes weeks, shadow deployment is not reproducible, and public benchmarks diverge from production in language mix and codebase structure.
Why it mattersPublic coding benchmarks do not resemble your production traffic; this is a method for building ones that do.
articleSmriti Jha, Matteo Paltenghi, Chandra Maddila, Vijayaraghavan Murali, Shubham Ugare, Satish Chandra
How well LLM-based test generation techniques perform with newer LLM versions?
Techniques for post-processing LLM-generated unit tests reported large gains in compilation and coverage on older models.
Why it mattersScaffolding tuned to older models can become dead weight — or a regression — when the model underneath changes.
articleMichael Konstantinou, Renzo Degiovanni, Mike Papadakis
SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch
Building a program from scratch given only documentation and an execute-only binary defeats frontier models, which solve under 1% of ProgramBench instances.
Why it mattersAgents are far weaker building from nothing than editing an existing repo, and spec elicitation is a concrete lever on that gap.
articleYihao Chen, Shi Chang, Feng Lin, Khaled Chawa, Boyuan Chen, Shaowei Wang, Ahmed E. Hassan
Impossible to hide secret ...: Uncovering Security and Privacy Issues in LLM-native IDEs
LLM-native IDEs are software systems and carry vulnerabilities like any other.
Why it mattersThe agentic IDE holding your credentials and source is itself a vulnerability surface.
articleMostafijur Rahman Akhond, Md Afif Al Mamun, Gias Uddin, Song Wang
ExplainBench: Evaluating Code Explanations from Agents
Agent changes now span tens to hundreds of lines.
Why it mattersWhen agent diffs outgrow manual review, the explanation is the review — and it needs to be measured.
articleZhiyuan Pan, Sungmin Kang, Imam Nur Bani Yusuf, Abhik Roychoudhury
Where Is the Cost of Third-Party API Routers in Agentic Software Development?
A third-party API router sits on the trusted path between a coding agent and its provider, able to inspect and modify every request and response.
Why it mattersMost agent stacks route through a router without accounting for what that layer costs or sees.
articleDonghao Fu, Jingxin Li, Xue Jiang, Yihong Dong
Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models
Self-repair is normally compared against not retrying at all, which confounds the value of the feedback with the value of a second attempt.
Why it mattersIf the retry, not the feedback, is doing the work, self-repair loops are spending tokens for nothing.
articleYuvraj Verma
Do Code Language Models Use Tests? A Behavioral and Representational Study of Test-Driven Code Generation
Public tests are used to steer code generation, but it is unclear whether models treat them as executable specifications or just extra context.
Why it mattersIf models are pattern-matching on tests rather than treating them as specs, test-in-prompt strategies need rethinking.
articleYunhao Liang, Chengguang Gan, Ruixuan Ying, Hanjun Wei, Zhe Cui, Shiwen Ni
Choosing Where and How to Moderate: End-to-End Trade-offs in Filter Placement and Response Rewriting
Moderation classifiers are evaluated in isolation, but deployment means choosing where to intervene and what happens after a flag.
Why it mattersIt reframes moderation as a placement-and-recovery design problem rather than a classifier-accuracy problem.
articleMengya Hu, Susie Park, Suzana Ilic, Qiong Wei, Sandeep Atluri, Myra Deng, Tucker Fross, Curt Tigges
Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes
Lossy verification speeds up speculative decoding by relaxing strict distributional matching, but that silently rewrites the decoding distribution.
Why it mattersTeams enabling lossy speculative decoding for throughput may be changing model outputs without realising it.
articleTianyu Wang, Yuxuan Zhou, Wenbin Wang, Heng Li, Zikai Xiao, Junyuan Shang
DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search
State-of-the-art retrieval models increasingly train on closed data, leaving a reproducibility gap.
Why it mattersAn open training recipe for retrieval models lets teams build code and long-context search without depending on closed data.
articleRapha\"el Sourty, Antoine Chaffin, Paulo Roberto Moura Junior, Am\'elie Chatelain
APEX-Accounting
APEX-Accounting, built by Mercor with Ramp, tests whether frontier models can do real accounting work.
Why it mattersHeld-out, real-work benchmarks are a better read on agent capability than saturated academic suites.
articleJulien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen
(Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding
Fifty-four students build a website with either an agent that edits their code or a chatbot they copy from.
Why it mattersIt puts evidence behind the suspicion that delegating edits to an agent quietly erodes the reviewer's grasp of their own codebase.
articleNishant Balepur, Connor Baumler, Valerie Chen, Eunsol Choi, Rachel Rudinger, Jordan Lee Boyd-Graber
OptimismBench: Forecasting Bias and the Alignment Effect in Language Model Judgment
When a model rates a startup's success at 70% and its failure at 15%, the missing 15 points are a distortion no calibration metric flags.
Why it mattersIf LLM judges tilt optimistic in a measurable direction, every eval and scoring pipeline built on them inherits that tilt.
articleSeonglae Cho, Adriano Koshiyama
BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms
A controlled scaling study across 28 nested corpus tiers spanning roughly 450-fold, holding questions fixed, comparing lexical, dense, graph and agentic retrieval.
Why it mattersRAG architecture choice is usually made on vibes; this puts accuracy-versus-cost scaling behind it on comparable footing.
articlePengyu Wang, Benfeng Xu, Shaohan Wang, Xin Zeng, Huarui Wu, Lei Zhang, Licheng Zhang
Steering Instruction Hierarchies at Inference Time
Instruction hierarchies assume a system prompt overrides a user or tool, and frontier models routinely break that.
Why it mattersInstruction-hierarchy violations are how injected tool output hijacks an agent; a training-free fix is deployable without retraining.
articleSiqi Zeng, Sewoong Lee, Han Zhao, Julia Hockenmaier
Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability
Deployed agents increasingly keep long-term memory as a directory of markdown files they read and reorganise with ordinary file tools, yet the medium's two working assumptions have gone untested.
Why it mattersFilesystem memory is the de facto default for long-running agents, and this is a rare systematic look at how to organise and evolve it.
articleSizhe Zhou, Sheldon Yu, Hui Wei, Junda Wu, Siru Ouyang, Yizhu Jiao, Shijia Pan, Julian McAuley, Yu Zhang, Tong Yu, Jiawei Han
Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems
Multi-agent systems in mixed-motive settings invite strategic deception under asymmetric information.
Why it mattersIt names and measures a failure mode that shows up as soon as agents in a system do not share objectives.
articleMarylou Fauchard, Florian Carichon, Margarida Carvalho, Golnoosh Farnadi
AI Security Priorities: A Field-Wide Agenda
A prioritised agenda for AI security drawn from structured interviews across industry, government and civil society and refined in a multi-sector workshop, aimed at the widening gap between AI adoption and AI security readiness.
Why it mattersIt maps where the gap between AI adoption and AI security readiness is widest, which is the agenda teams securing agents will be measured against.
articleGil Gekker, Rachel Steratore, Everett Smith, Asher Brass-Gershovich, Varun Gandhi, Nicole Nichols, Vijay Bolina, Buck Shlegeris, Lisa Einstein, Dan Lahav, Omer Nevo, Sella Nevo
MCP Tasks (async): Why Aren't Any Agents Supporting Them? — Cornelia Davis, Temporal
You invoke a tool and expect an answer, but real work takes time, and over that time connections drop, networks blip, and processes crash.
Why it mattersLong-running tool calls fail in exactly the ways distributed systems always have, and there is already a spec for it going unused.
videoAI Engineer
MCP Apps: Extending the Frontier — Ido Salomon & Liad Yosef
Chat and coding assistants return walls of text where a button, a chart or a small interactive view would say it faster.
Why it mattersIf tool calls can return interfaces instead of text, the shape of what an MCP server can usefully do changes.
videoAI Engineer
Can AMD break the CUDA Moat? AMD Advancing AI 2026
SemiAnalysis revisits its AMD call, from giving it a 0% chance of closing the gap with NVIDIA to a materially better one, and assesses whether AMD can break the CUDA moat.
Why it mattersWhether AMD is a real option changes procurement and portability decisions for inference and training.
articleBryan Shan
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
SemiAnalysis on Vera Rubin NVL72, the second generation of NVIDIA's rack-scale Oberon architecture, where engineering samples running DeepSeek R1 show 5.4x performance per megawatt and 5x per dollar over GB200 NVL72.
Why it mattersPer-dollar and per-watt inference deltas of this size reset the build-versus-rent calculation.
articleAlec Ibarra
tau2-bench v1.0.1 — τ-bench 1.0.1 — banking_knowledge Grading Fixes
A τ-bench 1.0.1 release fixing banking_knowledge grading, where a systematic penalty had zeroed rewards for correct-but-cautious agent behaviour.
Why it mattersPublished tau-bench numbers shift, and the fixed grader stops punishing agents for being appropriately cautious.
repobenshi34
TokenBudgeting: Our Conversations with Enterprises on Token Spend
Reports say enterprise token consumption has hit a budgeting wall.
Why it mattersHow enterprises are actually budgeting token spend sets the ceiling on what agent deployments get funded.
articleCrystal Huang
An index of the vibe-coding frontier. Corrections welcome.