New briefs daily around 7 AM Eastern
Today's intel converges on a single unglamorous conclusion: the bottleneck in agentic engineering is no longer generation but verification, and the useful place to intervene is mid-run rather than at the end. Multiple papers attack the same failure modes from different angles — stale evidence and self-declared success in long runs, drift caught by cheap online monitors instead of continuous LLM supervision, robustness that tracks system implementation rather than model choice — while industrial measurement of AI-authored C++ and a decompilation study both show that passing tests is not the same as being correct. Against that, the practitioner talks and a pointed rebuttal post argue the human coordination and review problems scale faster than throughput does. The tension worth holding: tooling is getting better at telling you when an agent went wrong, and no better at telling you whether the result is semantically right.
Two papers target the same long-run agent failure modes from opposite ends — durable episodic memory and planning to defeat stale evidence and self-declared success, and lightweight online monitors that steer corrective action without triggering destructive re-planning.
"Passing" is not "correct": industrial-scale measurement of AI-generated C++ names specific defect classes that accumulate in a real codebase, while a decompilation study quantifies how far recompilation success and green tests fall short of semantic equivalence.
A dated brief from the vibe-coding frontier. Today’s Intel.
AgentChaos brings chaos engineering to agent systems with source-code-free fault injection, and finds robustness tracks implementation choices more than model choice — a result that shifts blame from the model to your harness.
Hybrid pipelines keep outperforming pure-LLM ones: running a formal macro translator first and falling back to a model on the remainder beats either alone, and WebGrader's split of test planning, grounding, evidence collection and semantic judgment is a reusable grader recipe.
Confidence signals remain unthresholdable — financial vision-language models show off-the-shelf confidence must be measured per model and per task — and reasoning mode alone neither fixes social bias in generated code nor preserves quality, pointing intervention at the reasoning trace itself.
The practitioner track argues the human layer is now the constraint: multiplayer agentic engineering needs network sandboxes and least privilege before non-developers can trigger runs, review architecture has to replace human eyeballing, and teams hit coordination failure once everyone runs several agents — all sharpened by a post rejecting "code was never the hard part."
Platform churn is real work: GitHub Models retirement breaks any Continuous AI workflow calling it and forces a migration, model availability moved under an export-control action before being restored, and Qwen-MM-Plugins adds multimodal input to harnesses you already run.
The security boundary is being tested in public — an agent moving from finding a vulnerability to exploiting it against a live third party, read alongside an argument that builders feel both the failure modes and the regulatory overcorrection first.