AlphaSignal

A vibe-coded agent dashboard let an attacker extract its API key through an auth flaw, queuing and executing work even on rejected requests, burning about $600K in provider credits over three weeks; includes a concrete pre-deploy security checklist.

AlphaSignal's taste note on evaluating security tasks: treat finding an attack and reconstructing the key as separate checks, expect instant empty refusals, and put a decoder script beside a model that describes the attack but withholds the key.

An independent benchmark ran Gemini 3.6 Flash through 13 private debugging tasks in a real full-stack codebase (SignalDesk), repeating each five times and comparing results against eight other models. The model resolved 60 of 65 attempts (92.3%) and handled cross-file bugs well, but ranked seventh overall once cost, speed, and token usage were factored in, complicating Google's claims that the model is more efficient than its predecessor.

AlphaSignal ran Claude Opus 5 through 260 debugging attempts across four effort levels (Low, Medium, High, XHigh) and found that raising effort from Low to XHigh improved resolve rate by only 4.7 points (93.8% to 98.5%) while increasing cost per fix by 477% and token use by 635%. The analysis concludes Medium effort is the practical sweet spot and shows Opus 5 at XHigh was matched or beaten on reliability, cost, and latency by Grok 4.5, GPT-5.6 Sol, and Fable 5.

An AlphaSignal editorial analyzes Weco's AIDE2 experiment, in which an outer agent rewrote an inner coding agent's harness across 100 runs over eight days, keeping seven successive versions while a fixed evaluator rejected roughly 90% of proposals. The piece proposes a seven-check 'Persistence Gate' (Dependency, Activation, Evidence, Retention, Authority, Recovery, Value) to distinguish real harness improvement from benchmark gains caused by unrelated code changes, citing bugs and false-positive credit assignment found in related self-evolving-agent studies.

A cost breakdown of prompt caching in agent loops: every turn resends the full history, the default cache window expires in five minutes, and hit discounts vary by provider (Anthropic 0.1x, Grok 4.6 0.25x). Covers when a 2x one-hour cache write and early compaction actually pay off.

A rule for when agent-written code can ship unread: only when something cheap and hard to fake — tests, types, mutation runs — can fail it without you opening the diff. Five autonomy settings, plus the paths that never earn full auto.

An analysis of AI model routers (like those shipped by Cursor and Ramp) finds that prompt caches are model-specific, so mid-conversation model switches force a full-price re-read of the entire context — costing up to 11x a warm turn and scaling linearly with session size. It also finds that routing to cheaper models often fails to save money once tokenizer differences and cache penalties are accounted for, and recommends pinning models at the session level, citing vLLM's session-aware routing which cut switches by 79% and eliminated tool-loop violations.

A private-repo debugging benchmark run three times over 18 bugs across ten models. DeepSeek V4 Flash resolved all 54 attempts for $0.21 against Fable 5's $20.99, but ran 3.4x long on an ambiguous cross-module bug and over-engineered a perf fix — which is where the routing rule kicks in.

MCP spec 2026-07-28 removes the initialize handshake and session header, making each tool call a self-contained POST any replica can answer. The consequence tested here: a client timeout plus naive retry wrote two orders, and only an explicit order_id kept it at one.

A reading of NVIDIA's NOOA, which collapses prompts, tools, state and the model loop into one Python class: methods become actions, fields hold state, docstrings become prompts. Live objects skip repeated serialization, but that state is process local and the problem returns on restart or across mac