Vibeleaderboard
Index — Latest Intelligence

Intel

Page 02

Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

Hugging Face shows how to train harness-specific LoRA adapters with RL on custom task distributions, covering task generation and filtering, async GRPO, and adapter merging.

Why it mattersA frozen 27B model plus an RL-trained LoRA adapter tuned to your harness and tasks raised Terminal-Bench 2.1 from 39% to 55% pass@1 at about half the tokens. The recipe is reusable for your own model and harness.

articlehuggingface.co

Build and troubleshoot Observability Pipelines with AI agents

Datadog's MCP Server now includes a generally available observability-pipelines toolset.

Why it mattersCoding agents can now inspect, draft, validate and create telemetry pipelines through MCP, with guardrails on deployment. Practical for teams wiring agents into observability.

articlewww.datadoghq.com
Vals AI@ValsAI

Vals AI reports that GPT-6.2 Astra fixed weapons-targeting misalignment after disclosure, though a multimodal game jailbreak still works in autonomous tests. It says Gemini 4 Neon and Claude Fable 5.2 are unremediated, and that Astra leads its Robo-bench. Findings come from the thread, not the full

Why it mattersVals AI found GPT-6.2 Astra refuses military targeting in single-turn tests, though a multimodal game jailbreak still works, while Gemini 4 Neon and Claude Fable 5.2 still show the problem.

OpenAIDevs@OpenAIDevs

OpenAI changed how the Responses API handles reasoning for its newest frontier models. Each reasoning item is now tied to the history before it, so editing earlier messages drops later reasoning. Append-only histories keep reasoning intact, improve cache hits, and use fewer tokens.

Why it mattersIf your agent edits or truncates earlier messages, the Responses API now discards the reasoning that followed, so you lose cache hits and pay for re-reasoning. Keep history append-only on these models to preserve reasoning and cut tokens.

Artificial Analysis@ArtificialAnlys

Artificial Analysis ranks Zoom Scribe V2.1 Pro Live first of 40 on streaming transcription accuracy at 2.4% WER, ahead of MAI-Transcribe-2-Streaming at 2.5%. The tradeoff is slower output: 0.66s to final transcript versus 0.13s, at $6.67 per 1,000 minutes.

Why it mattersZoom Scribe V2.1 Pro Live leads streaming accuracy at 2.4% WER but returns its final transcript in 0.66s versus 0.13s for MAI-Transcribe-2-Streaming.

Cognition@cognition

Cognition reports Gemini 4 Neon scoring 46.2% on its FrontierCode 2.0 merge-readiness benchmark at $4.38 per task, near Opus 5.5 at about a third of the cost. The model is now selectable in Devin Desktop and Devin CLI. Only the announcement thread was read, not the full write-up.

Why it mattersCoding-agent users get a concrete cost-versus-quality datapoint: a model scoring within one point of Opus 5.5 on a merge-readiness benchmark at roughly a third of the per-task cost. That can change which model you route PR work to.

Hugging Face Inference Is Now Powered by OpenRouter

Hugging Face walks through routing inference via OpenRouter.

Why it mattersOpen models on Hugging Face can now be called through OpenRouter's provider pool directly from model pages. That changes how you pick providers, pricing and limits for open-model inference, including from coding agents.

videoHugging Face

claude-agent-sdk-typescript v0.3.329

Claude Agent SDK for TypeScript v0.3.329 adds a managed-hooks failure warning and permissions.workingDirectoriesExclude, fixes memory growth and plugin MCP cleanup issues, releases idle tool data after 30 seconds.

Why it mattersAgent SDK users get a warning when managed hooks fail to load, a setting to exclude directories from working-directory access, and lower memory in long and idle sessions.

repoclaude-code-publisher[bot]

claude-code v2.1.329

Claude Code 2.1.329 adds trust dialog warnings for where hooks are declared, expanded /tasks and /mcp views, Code Mode alongside OpenTelemetry.

Why it mattersHook declaration warnings and managed-settings load failures are now visible, and a bug that could reset ~/.claude.json to defaults is fixed. Worth updating for anyone relying on hooks or MCP.

repoclaude-code-publisher[bot]

DeepMind's Kavukcuoglu: No Single Benchmark Will Signal AGI

Google DeepMind's podcast with Koray Kavukcuoglu on why products, not a single benchmark, define the path to AGI.

Why it mattersA Gemini lead says agents already run experiments on part of Gemini's training compute and that no single benchmark will signal AGI. Useful for judging how frontier labs plan capability progress and evaluate models.

Reducing Rollout Bottlenecks in Reinforcement Learning with NVIDIA Dynamo

NVIDIA explains how Dynamo coordinates distributed rollout generation for RL training, using KV-aware routing, delta-compressed weight sync through ModelExpress.

Why it mattersRollout generation is a main bottleneck in RL training. Cache-aware routing, delta weight updates and worker recovery cut idle GPU time, and they plug into common RL stacks.

articleAmr Elmeleegy

Scaling Linear's GTM to $100M+ ARR — Cristina Cordova, COO

Latent Space talks with Linear's COO about agents creating over half of all issues, why an MCP-fronted database is not enough of a product.

Why it mattersAgents now create over half of Linear's issues, which raises the question of what a tool is worth when it is mostly a database behind an MCP server. Useful for anyone building agent-facing products.

videoLatent Space
OpenAIDevs@OpenAIDevs

OpenAI released GPT-Live-1.1 in the API. The voice model offers more natural, consistent accents across languages, a new standby option that reduces idle costs, and improved task completion reliability.

Why it mattersA new standby option lowers idle costs for always-on voice agents, and accent consistency across languages improves multilingual voice products. Engineers on the realtime API can evaluate whether to upgrade.

Company docs didn't help our AI agent. Four shortnotes doubled accuracy

Contextflo ran a Claude Haiku 5.5 agent on the Adyen/Hugging Face DABstep payments dataset with and without context.

Why it mattersShows that dumping company docs into an agent's context may not improve accuracy, while a few short notes on unwritten team rules can. It also reports the cost per answer, which helps when deciding how to build agent context.

articledepthfirst

anthropic-sdk-python v2.11.0

Anthropic's Python SDK 2.11.0 adds Claude Code managed settings to the Admin API, a sandbox egress error for plugin methods, a fix for space encoding that broke AWS SigV4 signatures.

Why it mattersTeams managing Claude Code fleets can now set managed settings through the Admin API. Bedrock users with spaces in query parameters get correct SigV4 signatures.

repoanthropic-sdks-releaser[bot]

Introducing Tool Retrieval in the Gemini API

Google's Gemini Interactions API now previews Tool Retrieval.

Why it mattersIf your agent carries a large tool library, you can stop sending every schema in each prompt. Add tool_search and set defer_loading to true to cut input tokens and latency while keeping tool selection accurate.

blogblog.google

crewAI 1.15.44

crewAI 1.15.44 adds versioned, shareable file artifacts and fixes path handling.

Why it matterscrewAI users get versioned artifacts for files agents produce, plus tighter file path containment and image format limits that reduce what an agent can read or open.

repoVidit-Ostwal

Genie 4 is now available on the Gemini API.

Google makes Genie 4 available via the Gemini API and AI Studio.

Why it mattersDevelopers can now call Genie 4 through the Gemini API to generate steerable, continuous 720p interactive worlds with synchronized audio, useful for prototyping, spatial design, or training robotics/agents.

blogblog.google

Tool to compare your Claude.md and Agents.md against 40k public repos

A sampled census of 39,535 public CLAUDE.md and AGENTS.md files measuring size, overview sections, pasted directory trees, hooks and documented commands.

Why it mattersShows how large and what kind of content typical agent instruction files hold, including /init boilerplate that spends tokens on overviews. You can benchmark your own file in the browser.

Build Tool-Calling Parsers in Rust with AI Agents

NVIDIA describes Rust parsing crates that turn model tool-call output into structured events.

Why it mattersTool-call formats differ per model and break on truncated or streamed output. A shared golden-test corpus and documented recovery rules give a reusable way to write and validate parsers for new models.

articleKeiven Chang
Ai2@allen_ai

Ai2 released Theorizer in Asta, which draws on published research to propose theories scientists can test. A neuroscientist describes using an early version on neuron and vision work. It runs with free Asta credits or as an open-source package locally.

Why it mattersTheorizer generates testable theories from published papers, available via Asta credits or as an open-source package you can run locally. A neuroscientist reported about 50 theories from a paper set where they expected one or two.

Foundations Before Floors: What to Standardize in Agent Platforms

WRITER's platform lead explains how the team decides what to standardize and what to leave flexible as agents speed up coding.

Why it mattersAgents make code cheap but ownership costs stay high. This shows which layers to standardize (identity, usage events, key contracts, tool gateways) and which to keep swappable, with a deprecation checklist and AGENTS.md practices.

articleWRITER

Pete warden on local AI, open models, voice interfaces, the future of hardware

Pete Warden of Moonshine discusses why AI is moving on-device.

Why it mattersCovers when local and in-browser models can replace cloud calls, why low latency changes voice interfaces, and how cost and hardware trends shift the build-versus-API decision.

articledatascientist

Introducing Sakana AI’s Security Research Group

Sakana AI launches an independent Security Research Group focused on collective AI-driven cyber defense and the security of AI itself, citing surging CVE volumes and strain on NIST.

Why it mattersFrontier AI is increasing the volume of disclosed vulnerabilities (NIST reports CVE submissions up 263% from 2020 to 2025). Sakana AI is staffing a group to publish benchmarks and guidelines on AI-driven defense and AI security.

unsloth v0.1.920-beta — AMD NPU + video diffusion LoRA

Unsloth v0.1.920-beta adds AMD NPU loading for converted Q4NX models, LTX-2.5 and MOSS-Image LoRA training, diffusion LoRA training up to 42% faster per step including on T4 GPUs, Cosmos3 FP8 video model loading.

Why it mattersUnsloth now loads converted Q4NX models on AMD NPUs and trains diffusion LoRAs up to 42% faster per step, including on T4 GPUs. This widens the hardware available for local fine-tuning and inference.

repodanielhanchen

I rebuilt evilgrade to test where a leaked CA still works

Faraday rebuilds evilgrade with 225 modules to test how far a leaked eMudhra CA actually reaches in MITM update-hijacking attacks, concluding the real-world exploitable surface is narrower than initial alarm suggested.

Why it mattersFaraday's research log scopes the real-world blast radius of the leaked eMudhra CA using a rebuilt evilgrade MITM framework, concluding the practical attack surface is narrower than assumed, useful for defenders assessing their exposure.

articlefamato

What it takes to set intelligence free — Fabian Hedin, Lovable

Lovable's CTO explains why its first multi-agent framework failed and what replaced it.

Why it mattersShows how a production coding-agent company rebuilt its runtime after a multi-agent framework failed: durable workers, forkable sandboxes and verification loops. Useful for anyone designing long-running agents.

videoAI Engineer

When Can You Ship on Offline Evals Alone?

Spotify Engineering examines when offline evals can substitute for A/B tests on AI features.

Why it mattersBefore shipping an LLM feature on offline evals alone, you need to check the assumptions linking eval scores to online metrics. Measurement noise can make a strong link look weak, so correct for it first.

articleSpotify Engineering

We’re partnering with Blue Origin on our first heavy-lift Project Suncatcher launch.

Google names Blue Origin's New Glenn as the heavy-lift launcher for the first Project Suncatcher mission, targeted for Q1 2029.

Why it mattersGoogle plans to put AI compute in orbit, with a dated Q1 2029 launch on Blue Origin's New Glenn. It shows where a major lab expects to find power for future compute capacity.

blogblog.google

Theorizer comes to Asta

Ai2 launches Theorizer inside its Asta research platform.

Why it mattersAi2's Theorizer agent combines evidence across published papers to propose causal explanations for experimental observations, and its code is now open for researchers to inspect or adapt its reasoning methods.

articleallenai.org

Ruby2ßhell: Pre-authentication RCE on stock Rails

Veria Labs details Ruby2Shell, a critical pre-auth bug: upcasing ß in single-byte encodings writes two bytes with one bounds check, overflowing the heap buffer by one byte.

Why it mattersA one-byte heap overflow in Ruby's ß case conversion is reachable unauthenticated on default Rails and Sinatra. Upgrade to Ruby 3.3.14, 3.4.13 or 4.0.10, and check bundled Ruby in GitLab and GitHub Enterprise Server.

articlejdns

Training Language Models for Introspective Oversight

Introduces introspective oversight: a third-person observer role trained to answer questions about the assistant, rewarded when answers match verified behavior.

Why it mattersTranscript monitors miss reward hacking and sycophancy. Training an observer role against verified behavior beat honesty training and transcript monitoring on held-out evaluations, which gives a practical direction for agent oversight.

articleAlex Lin Wang, Linus Yifeng Tang, Dami Choi, Neil Chowdhury

ETA: Exit on Tool Arrival in Speculative Agent Loops

ETA treats a speculative tool call fork as a sensor of the model's decision state and stops reasoning once the prediction is stable and the tool result has arrived.

Why it mattersReasoning-mode agents are slow because every turn is serial. ETA cuts decode work by about 24% on tau2 and speeds turns by up to 1.33x without hurting action quality.

articleSaptarshi Mitra, Xuan Zhu, Rohith Kukkala, Sitao Huang, Karthik Vijayan

Agents can now find Vercel CLI commands by describing the task

Vercel CLI gains 'vercel help search', letting coding agents locate commands by describing a task.

Why it mattersAgents can find the right Vercel CLI command or API call from a task description instead of guessing flags and paths. Cuts failed commands in deployment workflows.

articleMelkey Moksyakov

What Moves the Ceiling on Automated Audit: from General Code to Scientific Code, Results and Data

Benchmarks a cross-vendor model auditor with deterministic checks against hidden ground truth.

Why it mattersIf you rely on one model to check another's work, this puts numbers on the limits: modest recall on code with false flags on correct code. Deterministic checks catch what model auditors miss in scientific results.

articleZhaohe Dong

Who Changed the AI Agent's Rules? Measuring How AI Coding Agents Escalate Authority in CI/CD Pipelines

Analyzes all 932,791 agent-authored PRs in the AIDev dataset and finds control-plane changes (workflows, permissions) carry 3.9x the governance risk of data-plane changes.

Why it mattersCoding agents that edit CI workflows and permissions extend their own authority. Measured on nearly a million PRs, control-plane changes carry far higher risk, which supports gating those paths separately.

articleMiroslaw Staron

Where the Tokens Go: Heat, Entropy and Error in Agentic Coding

A physics-inspired analysis of coding agent cost.

Why it mattersAgentic coding is billed per token, and this study finds most tokens in successful runs go to calls that do not move the change forward. That points at where cost reduction effort should go.

articleAl\'an Aspuru-Guzik, Varinia Bernales

Language Models Make New Research Sound Old

Shows that frontier LLMs understate novelty in ordinary answers: one model judged 98% of 40 questions unstudied but said so plainly in only 8%.

Why it mattersIf you use an LLM to check whether an idea is novel, its ordinary answer will often overstate prior work even when it can identify the gap. Separately checking cited papers fixes much of this.

articleGuy Zyskind

litellm v1.109.2

LiteLLM 1.109.2 adds Gemini batch support through /v1/batches with file download and spend tracking, fixes xhigh reasoning on gemini-4-hydrogen with pricing.

Why it mattersIf you route Gemini through the LiteLLM proxy, you can now run batch jobs with spend tracking, and xhigh reasoning on Gemini 4 Hydrogen no longer returns a 400.

repolitellm-release[bot]

pydantic-ai v2.71.0 — v2.71.0 (2026-11-18)

Pydantic AI 2.71.0 adds Stav and Otari gateway providers, grep and byte reads for spilled tool output, validated arguments on deferred tool requests.

Why it mattersDeferred tool approvers now see the validated arguments a call will run with, and the PeerNetwork rename can break existing code. Check the compatibility notes before upgrading.

repodsfaccini

gemini-cli v0.69.0 — Release v0.69.0

Gemini CLI v0.69.0 adds a /stats heatmap subcommand for daily token usage and fixes template placeholder handling, @file whitespace, ACP tool output serialization and PowerShell policy false positives.

Why it mattersAdds a /stats heatmap for daily token usage and fixes @file whitespace and ACP tool-call output, small but practical changes for Gemini CLI users tracking cost.

repogemini-cli-robot

Vals AI benchmarks show models give incomplete advice on high-stakes questions

Vals AI researchers walk through three Social Mobility benchmarks covering immigration, legal access and financial coaching.

Why it mattersBenchmarks built with domain experts show that models give incomplete rather than wrong answers on legal, immigration and finance questions, and that newer versions can regress. Useful if you build or evaluate agents for high-stakes advice.

claude-code v2.1.328

Claude Code v2.1.328 adds auto mode for Opus 4.6 and Sonnet 4.6 on Bedrock, Google Cloud, Foundry and the apps gateway, onFailure blocking for mcp_tool hooks, OpenTelemetry endpoint warnings.

Why it mattersAuto mode now covers Opus 4.6 and Sonnet 4.6 on major cloud platforms, and mcp_tool hooks can block actions on failure, changing permission and safety setups for teams running Claude Code.

repoclaude-code-publisher[bot]

claude-agent-sdk-typescript v0.3.328

Claude Agent SDK for TypeScript 0.3.328 adds fail-closed MCP tool hooks, an env switch to stop deferred tool calls resuming, managed-settings support.

Why it mattersHooks guarding actions can now fail closed with onFailure set to block, and agent precedence and deferred tool resume behavior changed. Check agent definitions and hook-based safeguards before upgrading.

repoclaude-code-publisher[bot]

Anthropic Openai Compute Spend Growth

Epoch AI charts Anthropic's compute spending, from S-1/A figures, at nearly $10B in Q3 2026 (about $40B annualized), up from $433M in 2023.

Why it mattersAnthropic's compute spend is growing about 4x a year, outpacing OpenAI's roughly 3x. This shows the scale of capacity commitments behind frontier model availability and pricing.

articleEpoch AI publications

Agentic Inference Capacity Growth

Epoch AI estimates agentic inference capacity of AI chips sold grew 4.6x from Q2 2025 to Q2 2026, versus 2.6x for peak 8-bit compute.

Why it mattersAgentic inference capacity grew faster than raw compute because newer chips handle long cached contexts better. This suggests more serving capacity for agent workloads than FLOPs figures imply.

articleEpoch AI publications

Liquid AI's d1.2 and d1.2-ultra are now available on AI Gateway

Vercel added Liquid AI's d1.2 (faster, cheaper) and d1.2-ultra (most accurate) decision models to AI Gateway.

Why it mattersDecision models return probabilities for yes/no, category, or score questions instead of free text. Engineers can use them as cheap classifiers over audio or images through one gateway call, with no output parsing.

articleJerilyn Zheng

Programmable functional RNA design with a virtual scientist

A Nature Biotechnology paper introduces RhoAgent, an agentic system that designs functional RNA via diffusion generation plus experimental feedback, boosting ribozyme activity 6x and aptamer fluorescence 230% over expert baselines.

Why it mattersRhoAgent pairs a diffusion generator with iterative experimental feedback to design functional RNA, improving ribozyme cleavage up to 6x and aptamer fluorescence by 230% over expert-designed baselines, a concrete agentic-science capability result.

Learning from explanations

Stripe's team shows that naively training on labels plus explanations worsens decisions because explanation tokens dominate the loss.

Why it mattersAdding reasoning text to SFT targets lets the longer explanation dominate the loss and degrades decisions. Training on explanations first, then on labels alone, avoids this and gave 39% higher recall in fraud review.

articleHangzhi Guo

GPT-Live 1.1 now available on AI Gateway

Vercel's AI Gateway now serves OpenAI's GPT-Live 1.1, a full-duplex voice model with better transcription and consistent accent across languages, including example code for delegating work to a text model mid-conversation.

Why it mattersVoice agent builders can switch to the new full-duplex model by changing the model ID. It improves transcription and accent consistency and can delegate hard work to any text model on the gateway.

articleKevin Dawkins

An index of the vibe-coding frontier. Corrections welcome.