Vibeleaderboard
Index — Latest Intelligence

Intel

Page 77
AlphaSignal@AlphaSignalAI

A breakdown of agent security as three enforcement layers: sandboxed infrastructure holding credentials outside the agent, a reduced runtime surface with isolated ephemeral containers, and outbound network inspection that routes risky calls to an LLM judge and a human approver.

Why it mattersIt reframes agent safety as a systems problem with enforceable layers rather than prompt discipline, and names a concrete implementation at each layer.

See the top models on the Artificial Analysis Speech leaderboard: Vote for model

Cartesia's Sonic 3.6 takes the top of Artificial Analysis's controlled-voice arena at 1,117 Elo, ahead of Sonic 3.5, ElevenLabs v3 and Alibaba's Qwen-Audio-3.0-TTS-Plus.

Why it mattersCartesia's Sonic 3.6 now leads the controlled-voice arena at 1,117 Elo, ahead of ElevenLabs v3, at $49 per million characters — with the full table showing quality and price diverging sharply across providers.

articleArtificialAnlys
Artificial Analysis@ArtificialAnlys

Artificial Analysis puts Cartesia's Sonic 3.6 at number one on both the Provider Voice and Controlled Voice speech arenas, reporting Elo scores and appearance counts alongside per-character pricing and generation speed against ElevenLabs, Speechify and Alibaba entries.

Why it mattersAnyone shipping a voice agent gets current quality, speed and price for the leading text-to-speech models in one comparison instead of trusting vendor claims.

Aiden Bai@aidenybai

Aiden Bai released improve-threejs, an installable agent skill built on React Doctor that takes rough Three.js prototypes and rewrites them into faster scenes with the visual bugs cleaned up.

Why it mattersTargeted skills that repair a known class of agent-generated code are a practical answer to prototype-grade output, and this one covers a domain coding agents are visibly weak in.

Run this to install: npx skills add --skill improve-threejs Powered by React Doc

React Doctor scans a codebase deterministically for the patterns agents get wrong.

Why it mattersAgents produce React that passes review by looking plausible; a deterministic scanner installed as a skill catches state and effect misuse, performance traps and duplicated JSX before the code lands.

articleaidenybai
Cursor@cursor_ai

Cursor has launched Origin, its own code hosting platform, now rolling out in beta and wired directly into the editor. Teams get started by syncing existing repositories over from GitHub.

Why it mattersCursor moving upstream of the repository puts an agent-first vendor in control of review, CI and deploy hooks for teams already living in that editor.

We're rolling out the beta starting today.

Cursor now hosts code.

Why it mattersCursor now hosts repos and pull requests with two-way GitHub sync and agents operating in the same surface, so code hosting and agent execution stop being separate systems — with Vercel previews and Depot/Buildkite CI already wired in.

articlecursor_ai

Your agents can now pay for the APIs they use. We worked with AWS on the AgentCo

LangChain and AWS shipped AgentCore Payments middleware.

Why it mattersMachine-to-machine payment on the 402 path is the first credible answer to agents hitting metered APIs, and pairing it with a spend trace is the governance half teams get asked about.

articleLangChain

Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI

Across 11 context presets measured on an open source AI tutor, sending the full history beat every compaction technique on recall, cost and latency at once, because rewriting context invalidates the prompt cache that serves most of the tokens.

Why it mattersCompaction is a default in most agent frameworks, and this gives a cost model and a recall number for leaving the history alone instead.

videoAI Engineer

How canvases make agentic workflows visible, steerable, and cost-efficient

GitHub argues that chat is the wrong container for agent execution and describes canvases.

Why it mattersWhen agents produce changes faster than anyone can review them, chat transcripts stop being a usable record. Canvases give agent work a durable, steerable surface you can approve incrementally instead of auditing after the fact.

blogAyan Gupta
OpenAI Developers@OpenAIDevs

Numbers from OpenAI's GPT-5.6 builder guide: Luna held 98% of GPT-5.5's extraction accuracy at a eighteenth of the cost, programmatic tool calling cut one team's input tokens 21%, and retained reasoning plus compaction moved Sol from 13.3% to 38.3% on ARC-AGI-3 with roughly 6x fewer output tokens.

Why it mattersModel tiering, programmatic tool calling, and retained reasoning with compaction are producing order-of-magnitude cost differences on the same workloads — with published numbers you can check your own agent stack against.

Teaching Everyone to Fish for Tokens

Nathan Lambert compares open weight models to installed software versions rather than to Linux.

Why it mattersAnyone betting a product on open weights is choosing a dependency with a shelf life, and this frames what actually makes that ecosystem self sustaining.

articleNathan Lambert

Gemini 3.7 Flash Is HERE – Testing Google’s BEST Model Yet!

Bijan Bowen runs Gemini 3.7 Flash through practical coding and design tasks.

Why it mattersIndependent task by task footage of a new Flash tier model is the fastest way to see where it holds up on real coding work before you route traffic to it.

videoBijan Bowen
OpenRouter@OpenRouter

OpenRouter added a Cost per Session data layer: median real cost of tasks in each major harness over 30 days, which captures token efficiency that per-token pricing misses. Rankings are published and the data is queryable via a Dataset API under CC BY 4.0.

Why it mattersPrice per token hides how many tokens a model burns on a task. Cost per session gives median real task cost per harness over 30 days, queryable through a Dataset API, so you can compare models on actual spend.

From Ambient Documentation to Clinical Intelligence — Chaitanya Asawa, Abridge

Abridge's Chaitanya Asawa on why clinical AI evaluation resists a single correct answer.

Why it mattersRubric-based judging with independent authors and adjudication is a directly reusable pattern for anyone evaluating open-ended agent output where no single answer is correct.

videoAI Engineer

200 Million Patient Interactions Later — Vivek Muppalla, Hippocratic AI

Hippocratic AI runs 31 models on every clinical call, one holding the thread and 30 specialists in parallel, because a single model is a single point of failure.

Why it mattersIt is a rare, concrete account of designing a multi-agent system where latency and safety genuinely conflict, and the evaluation arithmetic transfers to any high-volume agent deployment.

videoAI Engineer
Bennett@b_nnett

A walkthrough of customizing the Codex CLI by having the agent read its own compiled bundle and inject new features, kept alive across updates with a mods folder the agent reapplies on request.

Why it mattersTreating a closed tool's compiled bundle as something an agent can patch turns unshipped features into an afternoon's work. The mods-folder reapply pattern is what makes those patches survive upstream updates.

Noam Brown@polynoamial

Codex can be pushed to a 1M-token window on GPT-5.6 Sol, but OpenAI's Noam Brown argues against it: the shipped limit is tuned for performance and cost, and autocompaction is meant to carry long sessions instead.

Why it mattersCodex's context limit is a deliberate cost/performance setting, not a ceiling worth raising by default — autocompaction is the intended mechanism, and the 1M option is opt-in for a reason.

A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing

A year-long production trace from a large inference provider, spanning popular and long-tail models, used to characterize how serving workloads evolve and what that implies for caching and load balancing.

Why it mattersAnyone sizing an inference tier or tuning prefix caching is currently guessing from short synthetic traces. A year of real production traffic, including long-tail models, gives that work an empirical baseline.

articleWilliam Nixon, Jon Durbin, Florian Standhartinger, Haryadi S. Gunawi, Juncheng Yang

Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency

An empirical pass over LLM-based program repair that pulls apart what actually drives outcomes.

Why it mattersRepair success is dominated by fault-localization precision and structural bug complexity, not model choice alone — and reasoning settings carry a cost that this study prices rather than assumes away.

articleJunchi Liu, Ali Bigdeli, Roya Daneshi, Atu Ambala, Sudipto Ghosh, Fabio Santos

Jais 2: A Family of Arabic-Centric Open Large Language Models

A second-generation family of Arabic-centric open large language models, released with a technical report covering the data, training recipe and evaluation behind the models.

Why it mattersArabic-centric open weights fill a real gap for teams serving non-English users where frontier models are weaker or unavailable. The training report also documents choices that transfer to other language-specific model efforts.

articleMohamed Anwar, Abed Alhakim Freihat, George Ibrahim, Mostafa Awad, Abdelrahman Sadallah, Gurpreet Gosal, Gokulakrishnan Ramakrishnan, Sarath Chandran, Biswajit Mishra, Rituraj Joshi, Ahmed Frikha, Etienne Goffinet, Abhishek Maiti, Ali El Filali, Sarah AlBarri, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Awantika Shukla, Sajid siddiki, Samta Kamboj, Onkar Pandit, Sunil Kumar Sahu, AbdelRahman Elbadawy, Amr Mohamed, Ahmad Chamma, Evan Dufraisse, Abdelaziz Bounhar, Dani Bouch, Hadi Abdine, Guokan Shang, Fajri Koto, Yuxia Wang, Zhuohan Xie, Ali Mekky, Rania Elbadry, Sarfraz Ahmad, Momina Ahsan, Omar El Herraoui, Daniil Orel, Hasan Iqbal, Kareem Elzeky, Mervat Abassy, Kareem Elozeiri, Saadeldine Eletter, Farah Atif, Nurdaulet Mukhituly, Haonan Li, Xudong Han, Aaryamonvikram Singh, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Larry Murray, Avraham Sheinin, Joel Hestness, Natalia Vassilieva, Hector Xuguang Ren, Zhengzhong Liu, Michalis Vazirgiannis, Preslav Nakov

Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation

RubricForge evolves an agent-judging rubric against ground-truth-labeled trajectories rather than hand-writing it, targeting the tendency of automatic judges to credit fluent but unsuccessful runs as successes.

Why it mattersOver-crediting is the main way agent eval suites lie to their owners, and hand-written rubrics are where it enters. Grounding the rubric text in environment reward is a directly copyable fix.

articleDarragh Quinn, David Dylan, Roisin Healy, Fionn Carroll, Maeve Donnelly, Cormac Sheehan

Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems

Names token inflation, the gap between a model's per-call price and the true cost of a retrying agent workflow, measures it across model tiers and task types, then builds a router that prices on the corrected figure.

Why it mattersPer-token price is what most teams budget from, and it can understate real agent cost by more than double on hard tasks. Measuring inflation per tier turns model selection into an arithmetic decision instead of a guess.

articleHeming Fu, Shan Lin, Qianqian Xie, Guojun Xiong

Building AI-Intensive Software with AI: Early Results and a Cautionary Tale on Measuring Development Cost

A six-person team built a RAG-based onboarding assistant under pervasive AI assistance and priced it three ways.

Why it mattersAI cost-savings ratios are easy to construct wrongly: pricing tokens under a flat subscription and mispricing the human counterfactual doubled this team's claimed 19.4x down to 9.9x on recheck.

articleVictor Barros de Miranda Neves, Kiev Santos da Gama, Vinicius Cardoso Garcia

Does a Language Server Save Tokens for Coding Agents? A Measurement Methodology and Preliminary Study

Tests whether language-server retrieval actually saves a coding agent tokens compared with plain lexical search, proposing tokens-to-success as the metric and an ablation that isolates the difference at equal task success.

Why it mattersEvery coding agent has to decide between grep and a language server for retrieval, and the choice is currently made on intuition. A tokens-to-success framing makes that decision measurable.

articlePengcheng Xu

AI Evaluation Should Work With Humans

A position paper arguing that measuring superhuman autonomous performance steers AI development toward replacing people.

Why it mattersMost agent benchmarks implicitly score replacement rather than collaboration, which shapes what gets optimized. Practitioners building evaluation harnesses should know the case against that default.

articleJan Kulveit, Gavin Leech, Tom\'a\v{s} Gaven\v{c}iak, Raymond Douglas

ATLAS: Discovering Agent Strategies through LLM-Guided Abstraction and Automata Learning

ATLAS learns interpretable state machines from agent execution traces, recovering the strategies an agent actually follows instead of reporting only task success and raw logs.

Why it mattersTrace dumps tell you what an agent did once; a learned automaton tells you what it tends to do. That is a practical handle for auditing agent behavior before trusting it with a task.

articleIgnacio D. Lopez-Miguel, Andreas Happe, J\"urgen Cito, Ezio Bartocci, Bettina K\"onighofer, Martin Tappler

Breaking Models to Test the Judge: A Mutation Testing Approach for Semantic Evaluators of Domain Class Diagrams

Rather than asking humans whether an LLM judge is right, inject known semantic defects and see if it catches them.

Why it mattersIf you rely on an LLM judge, mutation testing gives you a way to measure whether it actually detects defects instead of trusting spot-checks: inject known faults, score detection, compare to human validation.

articleKevin Delcourt, Meriem Ben Chaaben, Abdelhamid Rouatbi, Luciano Marchezan, Houari Sahraoui

DepWareTrans: Dependency-Aware Incremental Repository Migration across Co-executable Languages

Traces repository-scale code translation failures to dependency inconsistency between files.

Why it mattersAnyone pointing an agent at a whole-repo migration hits fragmentation across inter-file dependencies. Naming dependency inconsistency as the root cause suggests where to spend orchestration effort.

articleSivajeet Chand, Alexander Pretschner, Steve Haupt, Derui Zhu, Sushant Kumar Pandey

Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model

A monograph on operating coding agents as systems rather than models, locating reliability in the harness, execution state, retrieval, memory, permissions and review surfaces that surround the model.

Why it mattersTeams debugging agent failures reach for a better model first, when the defect is usually in the surrounding system. This gives a structured place to look and a vocabulary for evaluating the harness itself.

articleStephanie Jarmak

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings

A large study of reinforcement learning with verifiable rewards outside English, sweeping base models, training languages and reasoning-language rewards.

Why it mattersTeams doing RLVR post-training outside English have been extrapolating from English-only results. This says the native-language reasoning penalty is small, which changes how those training runs get configured.

articleKonstantin Dobler, Federico Scozzafava, Jonathan Janke, Mohamed Ali, Simon Lehnerer

LegacyWorld: Atomicity-Aware Evaluation of GUI Agents for Legacy Workflows

An evaluation of graphical-interface agents on stateful legacy workflows that scores atomicity, counting the invalid changes a failed run leaves behind rather than only whether the task eventually succeeded.

Why it mattersMost agent benchmarks score the end state and ignore the wreckage of failed attempts. For anyone deploying computer-use agents against real records, partial-failure damage is the risk that actually matters.

articleThilo Reintjes, Sivajeet Chand, Derui Zhu, Sushant Kumar Pandey, Alexander Pretschner

Modular Cognitive Architecture Emerges in Large Language Models

Circuit analyses across 46 tasks in four cognitive domains find that language models develop specialized regions echoing human brain organization.

Why it mattersIf task families recruit distinct neuron populations, that is a handle for interpretability work, targeted editing, and reasoning about which capabilities interfere. It reframes model internals as modular rather than uniformly entangled.

articlePengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda
Noam Brown@polynoamial

Codex exposes an opt-in 1M-token context window for GPT-5.6 Sol, newly documented after repeated requests. OpenAI tuned the default lower on purpose and says autocompaction is near-seamless — its own researchers advise against raising the limit.

Why it mattersCodex now documents an opt-in 1M-token context for GPT-5.6 Sol, but OpenAI's own researchers say the tuned default with autocompaction wins on both performance and cost.

A/B test models in production

Together's endpoints can now split live traffic across a control and up to 20 model variants, ramp a winner via blue-green rollout.

Why it mattersModel A/B splits move from feature flags and hash-mod routing in your application to the endpoint itself, so you can expose a candidate model to a chosen slice of real users and unwind it with one call.

articlewww.together.ai

Activity Dashboard

Usage analytics for teams running agents.

Why it mattersPer-agent, per-model and per-request spend, cache hit rate and tail latency are now queryable and groupable.

articleOpenRouter

Deploy Cursor Origin repositories with Vercel in public beta

Vercel opened public beta support for Cursor Origin repositories.

Why it mattersAnyone weighing Origin needs to know the deploy path already exists, and the speed of the wiring shows how fast the ecosystem is adapting to a week-old hosting platform.

articleCody Wong

GPT-5.6 Sol is 50% off on AI Gateway for the next month

Vercel is running GPT-5.6 Sol at half price on its AI Gateway through September 18, across every service tier including fast mode.

Why it mattersAnyone already sending Sol traffic through this gateway gets the lower rate with no code change, and the window closes September 18.

articleJerilyn Zheng

DeepSeek V4 Pro 0813 vs Claude Fable 5 on DeepSWE: Cost, Coding, and Routing

A DeepSWE comparison of DeepSeek V4 Pro 0813 and Claude Fable 5 that measures a 90x cost gap and an unusually low per-task correlation, the property that makes escalation routing between them worthwhile.

Why it mattersCross-model disagreement, not average accuracy, is what makes a cascade pay off, and this puts a number on it for two models at opposite ends of the price sheet.

articlewww.together.ai

$12B of US ratepayers' money wasted on a modeling mistake and PJM wants to do it again

SemiAnalysis rebuilt PJM's black-box Reserve Requirement Study and argues its methodology ignores winter efficiency gains and post-Elliott resilience, understating capacity by ~4GW.

Why it mattersPower in America's largest electricity market is procured off a model that understates existing supply by roughly 4GW, inflating capacity costs and squeezing the grid headroom datacenter expansion depends on.

articleRobert Boswall

Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

Hands-on notes on Qwen 3.8 27B, the Apache 2 licensed vision-capable release from Alibaba's Qwen lab, run in quantized form on a 128GB MacBook Pro and an NVIDIA DGX Spark.

Why it mattersA capable vision model in the 27B range runs on a well-specced laptop, but its default reasoning effort makes it impractical there; anyone trying it should turn that setting down before judging the model.

articlesimonwillison.net

Guillermo Rauch on Vercel's Shift Toward Agentic Infrastructure

In this interview, Vercel founder Guillermo Rauch describes how the platform has evolved from deploying front-end websites to hosting AI coding agents and autonomous software, driven by models like Opus 4.5.

Why it mattersRauch argues platforms must shift from human-facing elastic compute to 'agentic infrastructure' that lets coding agents deploy, secure, and maintain software autonomously over hours or days.

videoStanford Online

awesome-claude-code

A curated, community-maintained list of resources for Claude Code, covering guides, skills, agents, status lines, developer tooling, plugins, and orchestration patterns.

Why it mattersIt aggregates guides, skills, agents, plugins, and observability tools for Claude Code in one continually updated place, saving practitioners from hunting across scattered repos and blog posts.

repogithub.com

Awesome Hermes Skills

A curated, install-ready directory for the Hermes Agent ecosystem from Nous Research, cataloging the agent's 82 built-in skills, a 117-skill optional catalog.

repogithub.com

Awesome Claude

A curated directory of Claude AI resources for developers, aggregating official Anthropic documentation, SDKs (Python, TypeScript, Java, Go, Ruby, C#, PHP), agent SDKs, cookbooks, quickstarts, cloud provider integrations (Bedrock, Vertex AI, Azure), system cards.

articleawesomeclaude.ai

Awesome Claude Code

A curated, hand-maintained list of resources for Claude Code, organizing links to guides, skills, agents, status lines, plugins, orchestration tools.

Why it mattersIt aggregates guides, skills, agents, plugins, and observability tools for Claude Code in one continually updated place, saving practitioners from hunting across scattered repos and blog posts.

repogithub.com
Tibo@thsottiaux

OpenAI's Codex lead documents how to enable a 1M-token context window for GPT-5.6 Sol: three config.toml settings (model, context window, auto-compaction threshold) and a one-session CLI override. It also now works with ChatGPT account usage, with a caveat about the tuned default.

Why it mattersShows the three config.toml keys and a CLI override to run Codex with GPT-5.6 Sol at a 1M-token window, now available on ChatGPT sign-in as well as API keys. OpenAI warns the default is tuned for cost and performance.

Tuhin Srivastava (Baseten) on Production Inference Economics

Baseten is an AI inference infrastructure platform that runs production machine learning models—including custom and post-trained open-source models.

Why it mattersBaseten's CEO makes the case that once you are at scale, renting a frontier API stops being the cheap option and post-training an open model becomes both the cost play and the moat.

videoStanford Online

Anthropic and Chai Discovery on AI-Accelerated Drug Development

A Stanford guest lecture transcript featuring Anthropic's Eric Abrams and Chai Discovery's Josh Meier discussing how AI models are being applied across drug discovery and life sciences R&D.

Why it mattersExplains specific technical bottlenecks in applying AI to the 10-15 year drug development pipeline and how Anthropic's Claude and Chai's molecular design platform are targeting them.

videoStanford Online

Yash Patel on the History and Future of AI Model Training

A Stanford talk transcript in which Applied Compute founder Yash Patel traces AI model development from AlexNet through transformers, scaling laws, RLHF.

Why it mattersArgues the next AI capability bottleneck is continual, data-efficient learning from sparse real-world feedback rather than more pretraining, and explains why verifiable domains like code and math currently dominate RL training.

videoStanford Online

An index of the vibe-coding frontier. Corrections welcome.