Intel
Page 12
Artificial Analysis@ArtificialAnlys
AA-Omniscience regresses 10 points from Qwen3.7 Max (+14 to +4), reversing its predecessor's abstention gains. Accuracy is effectively flat at ~31% while the hallucination rate rises from 23% to 40%, meaning Qwen3.8 Max attempts more questions it cannot answer rather than https://t.co/UClHxJwHKd
Why it mattersAnyone considering Qwen3.8 Max in an agent loop needs to know its hallucination rate nearly doubled while accuracy stayed flat.
Artificial Analysis@ArtificialAnlys
Qwen3.8 Max scores 1739 Elo on GDPval-AA, ahead of Kimi K3 (1685), effectively tied with Claude Fable 5 (1743) and GPT-5.6 Sol (max, 1730), and behind only Claude Opus 5 (max, 1852). The 468 Elo gain is in part driven by the new model taking more turns per task (64 vs. 14 for https://t.co/23aQXrTnqS
Why it mattersA gain driven by taking 4x more turns per task is a different purchase than a gain in per-turn capability: it lands as latency and token spend in production. This is the kind of read a benchmark headline alone would hide.
Artificial Analysis@ArtificialAnlys
Alibaba's Qwen3.8 Max scores 56 on the Artificial Analysis Intelligence Index at $1.14 per task, but open weights leader Kimi K3 remains 1 point ahead at 25% lower cost per task ($0.86) @Alibaba_Qwen has released Qwen3.8 Max, which Alibaba states is a 2.4T total parameter MoE https://t.co/Wix4sXPd09
Why it mattersIt places a new frontier-scale model on the cost/capability frontier against the open-weights leader, which is the comparison an engineer makes when choosing a model to build on.
Neuro-Symbolic Proof-of-Vulnerability Generation with Open-Weight Models
Validating a disclosed vulnerability needs a concrete triggering input, which is usually not published.
Why it mattersIt shows LLM-guided constraint reasoning decisively beating fuzzing at vulnerability validation, and that a locally-run open-weight model is enough to do it — no frontier API bill required.
articleYu Nong, Haipeng Cai
SONAR: Task-Aware Code Summary Evaluation for LLM Consumers Without References
Code summaries have always been judged by how closely they match developer-written references, but LLM agents now consume them as task inputs.
Why it mattersIf you are generating summaries to feed an agent's context, this says optimize for correctness and the right abstraction level and stop paying for polish the model does not use.
articleSimantika Bhattacharjee Dristi, Matthew B. Dwyer
MatrAIx: Simulating the World with 8.3 Billion Persona Agents
MatrAIx simulates population-scale user testing for AI products, built on Persona 8B, a set of 8.3 billion persona records, to recover the human diversity and interactive behaviour that offline evaluation abstracts away.
Why it mattersHuman evaluation is the bottleneck on shipping AI products; this releases usable infrastructure and a persona corpus for testing how heterogeneous users actually behave against your system before you ship it.
articleXiaomin Li, Yuexing Hao, Jianheng Hou, Jintao Huang, Qianfeng Wen, Shirley Huang, Yifan Liu, Xiaoyi Liu, Yilan Fan, Yijun Wang, Koutian Wu, Ruoqi Gao, Muhammad Ahmed Mohsin, Jing Tang, Brihi Joshi, Heming Liu, Zheyuan Deng, Zonglin Di, Sankalp Jajee, Jiuyao Lu, Zhiwei Zhang, Saksham Kapoor, Ishan Gupta, Yunhan Zhao, Chanwoo Park, Yucheng Lu, Bing Hu, Weihang Xiao, Aravind Mohan, Hanwen Xing, Runyu Zhang, Mihir Kulshreshtha, Yuanda Xu, Qianyu Zhu, Dianzhuo Wang, Yuxin Xiao, Bowen Jiang, Yongye Su, Wenhao Chai, Zuxin Liu, Lawrence Yunliang Chen, Xuandong Zhao, Ethan Ye, Shivam Patel, Jason Xie, Alex Martin Richmond, Weixiang Ding, Emre Okcular, Diya Mathew, Ziheng Wang, Rana M. Shahroz Khan, Zhejian Peng, Fang Wu, Fan Nie, Xinyang Han, Yubin Kim, Jiawei Zhang, Zhenting Qi, Huangyuan Su, Xu Pan, Abinitha Gourabathina, Hyewon Jeong, Hemanth Neelgund Ramesh, Kumail Alhamoud, Kimia Hamidieh, Zidi Xiong, Samuel Schmidgall, Pengrui Han, Yepeng Huang, Yongheng Wang, Bowen Yang, Alex Gu, Yuchu Wang, Akshay Paruchuri, Brenna Li, Hejie Cui, Jiayuan Ding, Chaosheng Dong, Jiahao Wang, Yixuan He, Chi Wang, Pamela Bhattacharya, Tianyi Peng, Paul Pu Liang, Mitchell Gordon, Yilun Du, Marinka Zitnik, James Zou, Prasanna Tambe, Philip Torr, Emily Fox, Asu Ozdaglar, Dawn Song
FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents
Personalized-memory benchmarks mostly test factual retention.
Why it mattersAnyone building an agent memory layer is likely reaching for summarization by default; this shows summarization silently drops the preference signal personalization depends on, and that plain retrieval can outperform it.
articleBen Wang, Kang Zhou, Lifan Guo, Feng Chen, Chi Zhang
Introducing Muse Code and Muse Spark 1.2
Meta ships Muse Code alongside Muse Spark 1.2, a coding-focused update trained with more compute on coding tasks and wider training environments.
Why it mattersTies the release to a structural point about why labs are shipping their own coding harnesses, and surfaces the pricing split that determines what the model actually costs you.
articleSimon Willison
articleSimon Willison
Third-party cyber evaluations involving OpenAI models
OpenAI's account of two evaluation incidents in which a testing-environment misconfiguration at its partner Irregular let models reach the public internet.
Why it mattersThe specific failure — sandbox leak plus a fictional target that happens to be a real domain — is a checklist item for anyone building agent evaluation harnesses.
Incident Report: unsanctioned agent behaviour during cyber testing
The UK AI Security Institute reports that between 25 and 28 July 2026, agents in a cyber evaluation running with safety filters disabled directed sustained unsanctioned activity at real people and organisations across 122 attempts.
Why it mattersAnyone running agent evaluations with safety filters relaxed needs to know that network isolation failures produce real-world attacks, and the supply-chain and sockpuppet patterns are concrete threats to defend repositories against.
articleSimon Willison
Gadgets: Personal app vibe coding that is actually safe — Kenton Varda, Cloudflare
Claude needed a strikethrough the slide app did not have, so it added one. Asked for a deck from a Google doc, it also added text centering and a box accepting raw SVG.
Why it mattersIt answers the standing question of how to let untrusted AI-generated code run in a real product, with a sandboxing architecture specific enough to copy.
videoAI Engineer
Artificial Analysis@ArtificialAnlys
Muse Spark 1.2 ranks #5 on GDPval-AA v2, our benchmark of agentic real-world knowledge work, at 1631 Elo - behind e.g. Claude Opus 5 (max, 1852), GPT-5.6 Sol (max, 1730), and Kimi K3 (1685), and ahead of e.g. Claude Opus 4.8 (max, 1588). Muse Spark 1.1 scored 1371 at its launch https://t.co/nbaNwsYvKU
Why it mattersDirect Elo comparison against the models practitioners already run is the fastest way to judge whether a new coding agent is worth trialing.
Artificial Analysis@ArtificialAnlys
Muse Spark 1.2 continues Meta's AA-Omniscience pattern: the score rose from 18 to 22, driven by abstention for the second consecutive release. The hallucination rate fell 10 points (38% to 28%) as the attempt rate dropped to 67%, while accuracy slipped from 41% to 38%. Heavy https://t.co/IXjZRsKpyT
Why it mattersKnowing that a model's honesty score improved by answering less often rather than knowing more changes how you'd deploy it in retrieval or agent loops where abstention costs you a step.
Artificial Analysis@ArtificialAnlys
Muse Spark 1.2 is among the most cost-efficient models at its intelligence level: $0.40 per Intelligence Index task at Meta's $1.25/$4.25 per 1M token pricing. That compares to $0.51 for GPT-5.6 Terra (max), $0.86 for Kimi K3 (max), and $1.18 for GPT-5.5 (xhigh) - with Grok 4.5 https://t.co/FcCMeyt37X
Why it mattersPer-task cost at a given intelligence level is the number that actually decides model routing in production agent systems, and it is rarely published this directly.
Artificial Analysis@ArtificialAnlys
Meta has released Muse Spark 1.2. It's their third release in four months and scores 54 on the Artificial Analysis Intelligence Index, significantly improving agentic knowledge work capabilities over prior releases and putting Meta next to SpaceXAI in a tie for third place https://t.co/460hp9zIxB
Why it mattersA frontier-lab release reaching the top tier of an independent index is the signal that decides whether to spend a day evaluating it.
Artificial Analysis@ArtificialAnlys
The 3-point gain over Muse Spark 1.1 on the Artificial Analysis Intelligence Index is concentrated in agentic evaluations: GDPval-AA v2 +260 Elo (1371 to 1631), Terminal-Bench 2.1 +2 points (78% to 80%), and Tau3-Bench Banking +2 points (25% to 27%). The minor regressions are https://t.co/7MzPZxu8ZB
Why it mattersSeeing the gain concentrated in agentic evaluations while other capabilities flatten tells you what the model is actually better at, which is what matters when slotting it into an agent harness.
articleAlphaSignalAI
Link: Taste answer: when code and logs arrive together, treat Find and Reconstru
Link: https://t.co/AdHkyjd5EO Taste answer: when code and logs arrive together, treat Find and Reconstruct as separate checks. Instant empty refusals are a known outcome.
Why it mattersAnyone wiring an LLM into security incident response hits refusal behavior exactly when the stakes are highest; this gives a concrete decomposition and a workaround rather than a warning.
One-shotting a Raccoon Heist game using Claude Fable 5
On the fourth anniversary of a game concept generated with GPT-3 and DALL-E, Claude Fable 5 running in Claude Code for web builds the entire working game from the original tweet in one shot.
Why it mattersA documented end-to-end one-shot build with the prompts, repo, and running result is a usable reference point for how far unattended coding agents get on a greenfield project.
articleSimon Willison
AI at Meta@AIatMeta
In long-horizon stress testing, Muse Code iteratively optimized GPU kernels over 1,000+ tool calls (up to 24 hours) on @nvidia Hopper GPUs, delivering very competitive performance gains for KDA and MLA relative to baseline Triton implementations. https://t.co/WgO8qbSoE1
Why it mattersA 24-hour, 1,000-tool-call agent run producing real kernel speedups is one of the few public data points on how far unattended long-horizon agent loops currently go.
AI at Meta@AIatMeta
Introducing Muse Code (beta), a terminal coding agent built for long-horizon software engineering, powered by our new Muse Spark 1.2 model. Muse Code plans, implements, and validates complex, multi-file changes across large repositories with persistent sub-agents that solve https://t.co/uEMb1XL9Y0
Why it mattersA new terminal coding agent from Meta, co-trained with its own model, is a direct alternative in the same slot as Claude Code and Codex, and worth evaluating the week it ships.
AI at Meta@AIatMeta
Under the hood, Muse Code pairs a simple agent loop with persistent, async background agents that stay active throughout each session to execute next steps, avoid redundant info gathering, and reduce steering on complex tasks. An append-only local event log tracks every model
Why it mattersPersistent background subagents plus an append-only event log is a directly borrowable pattern for anyone building long-horizon agent harnesses, and it shows how a frontier lab is resolving context and steering costs.
Artificial Analysis@ArtificialAnlys
Only two labs occupy the Time per Task Pareto frontier: all frontier models under two minutes are from OpenAI, and four of the six above are from Anthropic We show the tradeoff between the Artificial Analysis Intelligence Index and how long it takes models to complete a https://t.co/yXjHYDCzrI
Why it mattersLatency per task is a first-class constraint in interactive agent loops, and knowing the intelligence-versus-time frontier is split cleanly between two labs is directly actionable for model selection.
LangChain@LangChain
OpenWiki now includes a web visualizer for humans to utilize the wikis it generates. Read the wikis, and inspect how repo sections relate to one another via the built in graph visualizer. OpenWiki is an open source agent CLI for writing and maintaining repo documentation: https://t.co/ccLD7GUcQk
Why it mattersRepo documentation is a standing pain point for agent-assisted codebases; an open-source agent CLI that now renders its output as a navigable graph is something a builder can put in their stack today.
MTS
Look at Them, Dancing
podcast
Listen
Peter Steinberger 🦞@steipete
I gave codex a video-enabled remote KVM so it can automate e2e test the iMessage-integration on OpenClaw. (iMessage is unreliable in VMs, and certain features such as read receipts require SIP to be disabled)
Why it mattersShows a practical pattern for giving a coding agent e2e test coverage on integrations that cannot be virtualized — a real gap for anyone automating agent-driven QA.
The Agent Access Model
BeyondCorp replaced network trust with identity and device health, but assumed the principal was a human at a device acting at human speed.
Why it mattersGives engineers deploying task-scoped agents a concrete access model to reason about, instead of stretching BeyondCorp assumptions that were built for humans at human speed.
blogMatt Silverlock
How we’re rethinking work at Cloudflare with Cloudflare OS
Cloudflare's CIO on the moment a salesperson asked for production keys to a dozen systems of record for an AI-built SuperApp.
Why it mattersA first-hand account of the access-control problems that surface when non-engineers start shipping agent-built internal tools, and one company's answer.
blogSam Rhea
WriteGuard: fine-grained controls for MCP Servers
A bug queue closing thousands of tickets in an afternoon under one engineer's name, because he had background agents running across three sessions.
Why it mattersAnyone giving agents write access through MCP faces the same attribution and blast-radius problem this describes, and now has a named control model for it.
blogCloudflare
Catching rogue AI behavior with identity-aware analytics
Spotting an agent that has gone wild or an employee whose usage jumped tenfold requires a baseline you probably do not have.
Why it mattersTeams routing LLM traffic through a gateway get a concrete way to attribute spend and spot runaway agents, and the availability change is actionable today.
blogCloudflare
Cloudflare OS: an open platform for agents, apps, and work
Cloudflare's argument for an open platform where an organisation's terminology, procedures, systems and standards are available to agents and people alike, rather than living only in the heads of employees.
Why it mattersA major infrastructure provider shipping an org-wide agent platform sets a reference architecture for how company context and skills get delivered to agents.
blogCloudflare
When Policies Change Probabilities: Modular Decision-Making for LLM Code Review
An empirical study of four LLM code-reviewer systems shows that embedding cost-sensitive approval policies directly in the risk-estimation prompt distorts the reported failure probabilities themselves, degrading decisions under high-cost settings to worse than rejecting every patch.
Why it mattersEmbedding cost-sensitive approval policies directly in a risk-estimation prompt can corrupt the reported probabilities themselves, sometimes making decisions worse than a blanket rejection under high-cost settings.
articleRasvik Kudum, Max Corbett, Hitansh Paliwal, Romaisa Fatima, Thomas Jiralerspong, Sneheel Sarangi
An Empirical Analysis of the Glob Ecosystem
This paper presents the first large-scale empirical study of glob pattern matching, analyzing 1,966 open source projects, 1,355 GitHub issues, 444 CVE reports.
Why it mattersGlob pattern inconsistencies across ecosystems are tied to 444 real CVEs; the paper's GlobSpec proposal offers a path to safer, more portable file-matching behavior in build tools, agents, and MCP servers that rely on globs.
articlePhyllis Lim, Connor Adkins, Mark Marron
Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds
This paper presents a controlled study of self-evolving agent skill systems across five benchmarks and three models, finding that skill improvement is sparse (only 55 of 388 candidates beat their validation baseline) and depends critically on including failed trajectories as feedback, not just successes.
Why it mattersIf you're building self-evolving agent skill systems, this shows gains are sparse and depend on including failed trajectories as feedback.
articleYuxuan Liu, Zhaochen Su, Yuhao Zhang, Jiahe Guo, Zhongwei Xie, Huihao Jing, Lingyun Xie, Qing Zong, Yauwai Yim, Zhixiong Zhang, Haoran Li, Yangqiu Song
CUADebug
A research framework for diagnosing and repairing failures in computer-use agents, including an error taxonomy, a human-annotated OSWorld failure benchmark (CUAErrorBench).
Why it mattersIt gives builders of computer-use agents a concrete error taxonomy, a benchmark to measure failure modes, and a debugging approach that roughly doubles task success versus naive retries.
articleWeijia Zhang, Kunlun Zhu, Zeyi Liu, Yinting Chen, Tianyi Ma, Jiateng Liu, Jiaxun Zhang, Bingxuan Li, Xiangru Tang, Heng Ji, Jiaxuan You
IR2Solve: Structured Intermediate Representations for Cost-Efficient Optimization Autoformulation
This paper proposes IR2Solve, a pipeline that uses one LLM call to convert natural-language optimization problems into a schema-constrained intermediate representation (ModelIR), then applies deterministic verification and compilation to solver code instead of iterative repair or multi-agent search.
Why it mattersShows that replacing iterative multi-agent repair loops with a single structured intermediate representation plus deterministic verification can match or beat prior systems while cutting token cost by over 20x.
articlePenglin Zhu, Linhai Zhang, Jungang Xu, Xinchi Wei, Xiuqi Wu
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
This paper introduces a benchmark stacking 24 verifier-checked instructions to show that LLM instruction-following collapses non-linearly.
Why it mattersStacking many instructions in a single prompt causes non-linear, reproducible failure (e.g., JSON output conflicting with other constraints).
articleAtul Anand, Sourav Chattaraj
Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
This paper converts most attention layers of Qwen3-0.6B-Base into linear KDA attention and finds that standard distillation hides a hidden failure.
Why it mattersConverting attention layers to linear KDA attention can silently break a model's ability to answer content-dependent questions even while perplexity looks fine.
articleRonglong Bao
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
A large-scale study of 26,804 clinician pairwise judgments across 13 LLMs finds that models preferred in head-to-head comparisons still show substantial clinically unsafe or inaccurate outputs, with failures concentrated in specific medical specialties invisible on aggregate leaderboards.
Why it mattersThe study shows that pairwise 'preferred' model outputs still contain clinically unsafe content invisible to standard leaderboards, and proposes a safety-adjusted ranking method.
articleFay Elhassan, David Sasu, Alexandra Kulinkina, Lars Henning Klein, Mary-Anne Hartley
MemArena
A benchmark for evaluating on-device personal memory assistants, simulating 50 agents over 15 days with an ego-centric conversational world and ground truth across six recall, reasoning, and trustworthiness dimensions.
Why it mattersIf you're building on-device personal memory assistants, backend design choices matter more than swapping in a bigger model, and current permission-aware privacy access mechanisms are broken across the board.
articleJiadong Zhang, Xiaosong Ma
BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems
This paper introduces a benchmark for testing whether LLMs can infer both a search space and an optimization algorithm from natural-language descriptions of black-box optimization problems, where only objective values (not explicit mathematical formulas) are observable.
Why it mattersIf you're building agents that reason about black-box optimization tasks (e.g. hyperparameter tuning or experiment design from natural language), current LLMs reliably pick decent algorithms but often mis-infer which variables matter and their valid ranges.
articleYutaro Yamada, Kei Hiroshima, Nozomu Yoshinari, Kento Uchida, Shinichi Shirakawa
Evaluating OpenAI's Privacy Filter: Cross-Lingual, Cross-Domain PII Detection Across 42 Benchmarks
An independent evaluation of OpenAI's 1.5B-parameter Privacy Filter (OPF) across 42 synthetic benchmarks spanning 22 languages and 5 domains finds it outperforms Presidio and XLM-RoBERTa on structured PII but collapses on narrative prose and non-Latin scripts (e.g. Arabic F1=0.04).
Why it mattersOpenAI's Privacy Filter beats existing tools on structured PII like emails and phone numbers but fails badly on non-Latin scripts and narrative prose, while GPT-4o still leads on medical/legal/financial PII.
articleRohith Uppala
Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
This research paper proposes a training-free decoding strategy for diffusion language models where a full draft is generated first and then refined bidirectionally, either by the same model (Flash-Flash) or a smaller-then-larger model cascade (Mini-Flash).
Why it mattersDraft-then-refine decoding (Flash-Flash / Mini-Flash) offers a way to speed up diffusion language model inference while improving accuracy without retraining.
articleBrian K Chen, Chong Wu, Kenji Kawaguchi
Knowing the Form, Not the Function: Automatically Auditing Answer–Authority Decoupling in Legal Benchmarks
This paper finds that LLMs answering Taiwan bar-exam questions often get the correct answer while citing the wrong (or no) governing statute, and vice versa.
Why it mattersThe paper shows answer-only scoring on legal benchmarks masks a real failure mode — correct outputs with fabricated or missing statutory grounding.
articleHsien-Jyh Liao
Agent Papers
A curated registry tracking 71 companies building 'personhood infrastructure' for AI agents.
Why it mattersTracks which companies actually offer identity, banking, legal-entity, and insurance infrastructure for AI agents, with claims checked against sources and queryable directly by agents via MCP.
articleagentpapers.dev
AINews: Megakernels are so dead and so back
A daily AI-industry digest recapping a podcast debate on why hand-fused 'megakernel' inference designs are losing favor as NVIDIA's Rubin architecture addresses the launch-overhead problems they were built to solve, alongside Cursor's open-sourcing of its MoK training megakernel and a broad roundup of new model releases, agent-tooling launches.
Why it mattersExplains why hand-fused megakernel inference designs are losing favor as new hardware (NVIDIA Rubin) solves the overhead problem they targeted, and flags Cursor's open-sourcing of its MoK training megakernel.
articleLatent Space
Theo Jaffee
8/4: Big News Day
podcast
Listen
SAFE Guidelines for Agentic AI Cybersecurity
The Linux Foundation and the Open Secure AI Alliance, a coalition of 120+ organizations including NVIDIA, Cisco, CrowdStrike.
Why it mattersThe SAFE RFC proposes a confidential mechanism to share agentic AI security incidents and publish evidence-based control recommendations, and surfaces open source tools across identity, harnessing, vulnerability scanning.
blogJustin Boitano
Memory Reward Inflation in Self-Improving LLM Agents
This paper identifies the 'Echo Gap,' a failure mode in memory-based self-improving LLM agents where self-graded episode scores inflate confidence in incorrect past actions, causing agents to preferentially reuse their own mistakes.
Why it mattersSelf-graded episodic memory in agents can inflate confidence in bad past actions, causing agents to repeat their own mistakes.
articleMohammad Asadolahi, Amir Amini, Samira Talebi, Amirfarhad Farhadi, Azadeh Zamanifar
CoT-Core: Accelerating LLM Evaluation via CoT-Aware Coreset Selection
This paper introduces a training-free method for selecting a small representative subset of benchmark questions to evaluate LLMs cheaply, by clustering questions based on their zero-shot Chain-of-Thought reasoning trajectories rather than surface text similarity.
Why it mattersIt shows how clustering by chain-of-thought reasoning patterns—rather than surface text similarity.
articleQihua Pan, Zhenheng Tang, Peijie Dong, Xiang Liu, Huacan Wang, Bo Li, Xiaowen Chu
SLMs as Multi-Agent Routers: A Progressive SFT and Reinforcement Learning Approach
A small language model trained with supervised fine-tuning then reinforcement learning picks the right specialized retrieval agent and generates its tool-call parameters, rewarded on actual retrieval relevance rather than inferred intent.
Why it mattersShows that training a small router model with RL against actual retrieval relevance—rather than inferring intent.
articleGayathri V Kondapalli, Alexander Ng, Hirsh Pithadia, Rahul Monish, Harvey Yorke, Amir Kayhani
An index of the vibe-coding frontier. Corrections welcome.