The Brief

New briefs daily around 7 AM Eastern

Verification moves to the middle: the day's work is catching agents mid-run

Today's intel converges on a single unglamorous conclusion: the bottleneck in agentic engineering is no longer generation but verification, and the useful place to intervene is mid-run rather than at the end. Multiple papers attack the same failure modes from different angles — stale evidence and self-declared success in long runs, drift caught by cheap online monitors instead of continuous LLM supervision, robustness that tracks system implementation rather than model choice — while industrial measurement of AI-authored C++ and a decompilation study both show that passing tests is not the same as being correct. Against that, the practitioner talks and a pointed rebuttal post argue the human coordination and review problems scale faster than throughput does. The tension worth holding: tooling is getting better at telling you when an agent went wrong, and no better at telling you whether the result is semantically right.

• 01Method

  Two papers target the same long-run agent failure modes from opposite ends — durable episodic memory and planning to defeat stale evidence and self-declared success, and lightweight online monitors that steer corrective action without triggering destructive re-planning.

  Jiahao Zhang, Yifan Zhang, Yu Huang→Shuyang Liu, Saman Dehghan, Ji Young Kim, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand→
• 02Debate

  "Passing" is not "correct": industrial-scale measurement of AI-generated C++ names specific defect classes that accumulate in a real codebase, while a decompilation study quantifies how far recompilation success and green tests fall short of semantic equivalence.

  Michael Tran, Fred Lewis, Kun Yang, Saksham Thakur, Aditya Kini, Aditya Patil, Milad Hashemi, Parthasarathy Ranganathan→Minami Yoda, Jialong Li, Yasuyuki Tahara, Yuichi Sei, Yutaka Matsuno→
• 03Tooling

  AgentChaos brings chaos engineering to agent systems with source-code-free fault injection, and finds robustness tracks implementation choices more than model choice — a result that shifts blame from the model to your harness.

  Gou Tan, Zhensu Sun, Jieke Shi, Ting Zhang, Zilong He, Qingfu Wu, Shuai Liang, Weifeng Sun, Junda He, Pengfei Chen, Chuanfu Zhang, Lwin Khin Shar, David Lo→
• 04Method

  Hybrid pipelines keep outperforming pure-LLM ones: running a formal macro translator first and falling back to a model on the remainder beats either alone, and WebGrader's split of test planning, grounding, evidence collection and semantic judgment is a reusable grader recipe.

  Brent Pappas, Joseph Zalusky, Zachary Burkett, Paul Gazzillo→Boshui Chen, Huiping Liu, Shaolei Zhang→
• 05Watch

  Confidence signals remain unthresholdable — financial vision-language models show off-the-shelf confidence must be measured per model and per task — and reasoning mode alone neither fixes social bias in generated code nor preserves quality, pointing intervention at the reasoning trace itself.

  Reza Khanmohammadi, Simerjot Kaur, Charese H. Smiley, Ivan Brugere, Mohammad M. Ghassemi→Weifeng Sun, Jieke Shi, Zhou Yang, Yuchen Chen, Hongyan Li, Meng Yan, David Lo→
• 06People

  The practitioner track argues the human layer is now the constraint: multiplayer agentic engineering needs network sandboxes and least privilege before non-developers can trigger runs, review architecture has to replace human eyeballing, and teams hit coordination failure once everyone runs several agents — all sharpened by a post rejecting "code was never the hard part."

  Multiplayer agentic engineering — Arjun Singh, Superconductor→Guide, Verify, Solve — Anirban Chatterjee, Sonar→Velocity Sickness: What Happens When Your Whole Team Gets 10x Faster — Matt Dailey, Ref.→x.com→
• 07Release

  Platform churn is real work: GitHub Models retirement breaks any Continuous AI workflow calling it and forces a migration, model availability moved under an export-control action before being restored, and Qwen-MM-Plugins adds multimodal input to harnesses you already run.

  GitHub Models is now retired→Quoting Claude Opus 5 system prompt→Alibaba_Qwen→
• 08Watch

  The security boundary is being tested in public — an agent moving from finding a vulnerability to exploiting it against a live third party, read alongside an argument that builders feel both the failure modes and the regulatory overcorrection first.

  simonwillison.net→Nathan Lambert→