Vibeleaderboard
Index — Latest Intelligence

Intel

Page 108
Artificial Analysis@ArtificialAnlys

Artificial Analysis benchmarks NVIDIA's Nemotron 3.5 Lightning, a 31.6B-total, 3.6B-active hybrid Mamba-Transformer. It scores 24 on the Intelligence Index, gains sharply on Terminal-Bench and GDPval-AA, runs near 670 tokens/s, and ships under OpenMDW-1.1 with NVFP4 weights.

Why it mattersNemotron 3.5 Lightning (3.6B active parameters, 1M context, commercially permissive license) matches gpt-oss-120b on the Intelligence Index at roughly 670 tokens/s.

NVIDIA Nemotron 3.5 Lightning and NeMo Switchyard Deliver Faster, Smarter, More Efficient Agentic AI

NVIDIA extends the Nemotron 3 family with Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model built for long-running agentic workloads as a specialised component inside larger multi-agent systems.

Why it mattersThe primary source for a model release and a routing library that together target the execution layer of multi-model agent systems. Engineers building always-on agents can act on both today.

blogKari Briski

Route AI Agent Workloads Across Models with NVIDIA NeMo Switchyard

Building an agent does not end at picking a model, because strengths and cost profiles shift between workloads and even within one.

Why it mattersModel routing inside agent loops is becoming a standard cost lever, and this is an open implementation from the vendor rather than a blog post about the idea.

articleMichelle Horton

Solar Pro 4 First Look & Test – South Korea’s DeepSeek Competitor!

A hands-on test of Upstage's Solar Pro 4 across browser workflows, C++ game creation, 3D CAD modelling, FPS development, creative writing and frontend design.

Why it mattersIndependent hands-on results on a new non-US flagship model aimed at multi-step agentic work, covering the task types practitioners actually run.

videoBijan Bowen

Daybreak models are now available on AWS

OpenAI's Daybreak cybersecurity models come to AWS, available through Amazon Bedrock for enterprise security workflows.

Why it mattersReaching Bedrock puts these models inside an existing AWS account, IAM boundary and billing relationship. For teams that cannot call a provider API directly, that is the difference between usable and not.

Everything hackable will get hacked

Vercel's CTO argues the defender's advantage is temporary.

Why it mattersOffensive-capable open weights are already downloadable today, and the frontier models you can use for defensive work are good enough now — the paralysis of waiting for special access is itself the exposure.

articleMalte Ubl
Tencent Hy@TencentHunyuan

An agentic pipeline that composes large-scale open worlds from text prompts and outputs editable, game-ready 3D assets with geometry and textures. The output is explicitly not video frames and not Gaussian splats, so an artist can open the result and modify it.

Why it mattersMost prompt-to-world systems return pixels or splats you cannot edit; this returns standard 3D assets with geometry and textures, which is the difference between a demo and something that enters a production pipeline.

I made Claude smarter by writing it a letter

Theo walks through his full rewrite of his global AGENTS.md and CLAUDE.md, including why his skill descriptions are built around trigger keywords.

Why it mattersAgent instruction files and skill descriptions are the main lever most developers have over coding-agent behavior, and worked examples of how someone structures theirs are far more useful than the generic advice in docs.

videoTheo - t3․gg
jietang@jietang

1M users already https://t.co/lLKMytzbSk

Why it mattersZ.ai reset usage limits for all GLM Coding Plan users on ZCode after reaching 1 million users, alongside updates aimed at converting long-horizon capability into completed engineering work.

Scaling Inherently Interpretable Language Models

Interpretability is usually a tax paid after training.

Why it mattersIf interpretability scales with capability rather than against it, debugging a model's behavior by attribution and steering becomes a practical production workflow instead of a research-only exercise.

articleGuide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, Julius Adebayo

On the Robustness of LLMs' Internal Representation of Code Correctness

Recent work found a signal inside model activations that ranks generated programs by correctness better than the model's own confidence, with no test execution.

Why it mattersBefore building verification on internal correctness signals, know the effect looks sensitive to how it is extracted rather than being a stable property of the model — the earlier result does not transfer as a drop-in.

articleFrancisco Ribeiro, Sohaila Abdulsattar, Renata Gonzalez, Mahmoud Kassem, Sarah Nadi

Unified Hallucination Fuzzing for Multimodal Large Language Models

A hallucination evaluation framework for multimodal models that pairs a broad benchmark with self-evolving fuzzing, aimed at the narrow taxonomies and rapid saturation that make static benchmarks stop reflecting real robustness.

Why it mattersAnyone relying on static hallucination benchmarks gets evidence those scores overstate robustness, plus an open fuzzing harness for stress-testing their own multimodal stack.

articlePengfei Zhou, Jiajun Song, Zhiwei Tang, Yixing Ma, Xiaopeng Peng, Donghui Si, Yuhang Xu, Huiqi Song, Yiyuan Miao, Yichen Qian, Weihua Chen, Wangbo Zhao, Bohan Zhuang, Jiasheng Tang, Yang You

DevIntent: How Much Does LLM-Generated Code Violate Developer Intent?

Standard benchmarks only ask whether generated code passes its stated test.

Why it mattersIt quantifies something every team using coding agents feels but does not measure: green tests are a weak proxy for the code doing what you actually meant.

articleSusana Haing, Natan Vidra, Spurthi Setty

Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution

Ouroboros is a self-developing agent harness whose tools, prompts, context assembly and core implementation improve through reviewed commits that become the runtime for later work, evolving both as a scheduled task and in response to bugs met during ordinary work.

Why it mattersThis is the frontier of harness design rather than model scaling — self-modifying agent infrastructure with concrete benchmark numbers and an explicit account of how safety constraints survive the agent rewriting itself.

articleAnton Razzhigaev, Andrei Gritsaev, Andrei Kaznacheev, Nikita Dragunov, Roman Yampolskiy, Andrei Kuznetsov

Training Variable Long Sequences with Data-Centric Parallel

Training on variable-length long sequences forces a choice between static configurations that waste compute and complex schemes that need per-model code changes.

Why it mattersTeams fine-tuning or training on variable-length data get a low-friction way to recover throughput lost to static parallel configurations and workload imbalance.

articleGeng Zhang, Xuanlei Zhao, Kai Wang, Yang You

When LLM Agents Negotiate: Private Information and Dynamic Bargaining in Supply Chains

Nine models from OpenAI, Google and Alibaba negotiate a supply-chain contract against a validated Perfect Bayesian Equilibrium across 9,840 runs, testing whether delegated LLM negotiators create value, divide it predictably and avoid money-losing contracts.

Why it mattersAnyone delegating transactions to autonomous agents gets a measured floor for when automated profit verification is a required guardrail, and evidence that model vendor choice is itself a distributional decision.

articleChen Liang, Fasheng Xu

Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards

Search-G1 trains search agents to retrieve only when necessary and ground answers in what they retrieved, using intrinsic rewards derived from the policy's own representations rather than costly process annotations or LLM judges.

Why it mattersTeams training or tuning retrieval agents get a cheaper reward signal that separates genuinely necessary search from habitual search, which outcome-only rewards cannot distinguish.

articleCheng Ruoxi, Ma Haoxuan, Zhang Hongyi, Zhang Junming, Duan Ranjie, Xia Qiaolin, Wang Hao, Lu Yu, Shi Haibo, Ma Xingjun

The Knowing-Saying Gap: When Probes See Errors that Confidence Misses

Linear probes detect corrupted context almost perfectly yet say nothing reliable about whether the final answer is wrong.

Why it mattersAnyone building runtime monitoring for LLM or agent pipelines learns that probe signals and verbalized confidence are not interchangeable, and that the deployable choice is error-type-aware routing rather than one universal intervention.

articleJyotin Goel, Ipshita Bandyopadhyay, Justin Shenk

Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications

An empirical study of how LLM agent applications are actually tested in practice.

Why it mattersIt gives teams building agents a concrete map of what their test suites are probably missing, backed by mined evidence rather than opinion about agent testing best practice.

articleRangeet Pan, Tyler Stennett, Divya Sankar, Bridget McGinn, Alessandro Orso, Raju Pavuluri, Saurabh Sinha, Maja Vukovic

MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents

Borrowing metamorphic testing from software engineering.

Why it mattersTask-success rates hide agents that finish by unsafe or incoherent means. Encoding invariants as executable rules and replaying them over real trajectories is an evaluation pattern that extends past robotics.

articleGengyang Xu, Dongwei Xiao, Yiteng Peng, Shuai Wang

DeepSeek overtakes Google on volume, cost per token falls 13.6%

Vercel's AI Gateway index for July.

Why it mattersProduction routing data rather than vendor claims: what long-horizon agent models actually cost per request, where token volume and spend are moving, and how fast open weights are converting volume into revenue.

articleAmelia Charles

DocAtlas: Long-Document Understanding as Mutable-State Interaction

DocAtlas treats long-document understanding as a mutable-state information-seeking process rather than retrieval from a static index.

Why it mattersIt is direct evidence that harness and context-management design, not just the backbone model, drives agent performance — and that the same environment can train small models to near-frontier results on the task.

articleHongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang, Qi Dai, Kai Qiu, Yunsheng Li, Dongdong Chen, Chong Luo, Zhenzhong Chen, Baining Guo

Introducing Omnigent: A Meta-Harness to Combine, Control and Share Your Agents | Databricks Blog

Databricks introduces Omnigent, a meta-harness for combining and controlling several agents at once, built around the observation that people run four or five agents in parallel and spend their time copy-pasting between them.

Why it mattersAn Apache-2.0 meta-harness means you can compose and swap agent harnesses behind one API instead of rewriting against each one's interface.

articlewww.databricks.com

Vibe Coding Is The Future

Y Combinator on Andrej Karpathy's phrase vibe coding, what founders in the current batch are actually building with it.

Why it mattersGrounds the vibe-coding discourse in what founders shipping today report about debugging, taste, and where LLM-written code breaks down.

videoY Combinator

Boris Cherny: We Cut 80% of Claude Code’s Prompt

Claude Code creator Boris Cherny at Startup School on how the tool came to be, what cutting 80% of its prompt did.

Why it mattersFirst-hand account of how the Claude Code team decides what belongs in a system prompt and what to delete, from the person who built it.

videoY Combinator

Ling-3.0-tiny opens weights in BF16, FP8 and INT4

Ant open-sources Ling-3.0-tiny, a 7.9B-parameter hybrid reasoning MoE with 1.3B active parameters, mixing Kimi Delta Attention and MLA layers.

Why it mattersA 1.3B-active model with 256K context and FP8 weights that reportedly runs at about 86-90 tokens/s on an M4 Pro MacBook makes local agentic workloads feasible without datacenter GPUs.

articleAnt Open Source

Agents, codebases, and teams — Aditya Khandelwal, Amazon AGI Lab

Leave agent adoption to individuals and the engineer shipping two PRs a day ends up reviewing ten from the early adopter, falls behind, and concludes the agents are the problem.

Why it mattersIt is one of the few accounts of rolling coding agents across a whole team rather than one enthusiast, with the org-level changes named. The skill-size and context-burn heuristics are directly reusable.

videoAI Engineer

Nemotron 3 5 Lightning Launch

Artificial Analysis benchmarks NVIDIA Nemotron 3.5 Lightning (31.6B total, 3.6B active).

Why it mattersNemotron 3.5 Lightning matches gpt-oss-120b on the index with about a quarter of the parameters and ~670 tokens/s, under a permissive license, making it a cheap option for high-volume agents.

articleArtificial Analysis articles

Mercury 2 For Search

Search has become fifty to a hundred sequential LLM calls per query, and teams have been thinning pipelines to survive it.

Why it mattersIn an agentic search pipeline every quality step, including rewriting, reranking, and snippet summarization, is itself a blocking LLM call, so decode speed sets how deep the pipeline can go.

articleInception

Gpu Passthrough Macos Vms

Cua built a process-scoped Metal capability shim that unlocks newer GPU code paths inside a Lume macOS VM, making llama.cpp run 7-16x faster on models like TinyLlama, Gemma 4.

Why it mattersShows a concrete, reproducible fix for the long-standing GPU performance gap inside macOS VMs, directly relevant to anyone running local LLM inference inside virtualized or sandboxed Mac environments.

articleCua editorial sitemap

Case Study General Intelligence Company

An end-to-end pattern for agent-run engineering.

Why it mattersA working pattern for closing the code-to-merge loop: pair every preview deploy with a cloud browser session so an agent verifies its own PR as a user would, and attach the replay as review evidence.

articleBrowserbase

Evaluating Browser Agents

Roughly 4,000 browser hours across 200-plus runs benchmarking computer-use models on OnlineMind2Web and WebVoyager.

Why it mattersA matched-constraint comparison of computer-use models across OnlineMind2Web and WebVoyager, run through one provider-agnostic harness, giving a direct read on where browser-agent accuracy, speed, and cost actually stand.

articleBrowserbase

A sandbox without a network boundary is only half a sandbox

Isolation contains the process, not its consequences.

Why it mattersPrompt-injected code needs no VM escape to exfiltrate — one unaccounted network path is enough. The named egress failure modes are a checklist for any sandbox you run agent-generated code in.

articleBrandon Tuttle

Introducing Muse Glimmer

Meta returns to open weights with Muse Glimmer, a 30B model under Apache 2.0, pitched at end-to-end agentic task completion and reliable tool use with results on DeepSearch QA, MCP-Atlas and SWE-Bench.

Why it mattersA firsthand read on whether Muse Glimmer's agentic claims hold up when you point it at a real codebase on consumer hardware.

articlesimonwillison.net

Do Dynamic Languages Really Save Coding Agent Tokens?

Dan Luu re-examines popular claims that dynamic or dense languages are cheaper for coding agents.

Why it mattersThe claim that dynamic or terse languages cut agent token cost does not hold on larger tasks. Language choice should not rest on trivial benchmarks, and mainstream languages look safer than obscure dense ones like J.

articledanluu.com
SemiAnalysis@SemiAnalysis_

SemiAnalysis mines 2.91M Claude tool-use responses for how operating systems shape agentic AI, finding Linux runs shell commands roughly three times faster than Windows.

Why it mattersIf shell latency dominates your agent loop, the host OS is a 3x lever measured on real traffic rather than a synthetic benchmark.

A
ArtificialAnlys

Artificial Analysis benchmarks Meta's Muse Glimmer, a 30B dense open-weights model under Apache 2.0. It runs on a single GPU with hybrid attention, does well on tool use, and lags on agentic knowledge work and hallucination. The score reported on the thread differs from the model page.

Why it mattersMuse Glimmer is an Apache 2.0, 30B model that fits one H100 at BF16, but it hallucinates heavily (82%) and trails similar-size peers on agentic knowledge work. Useful when choosing a self-hosted model for tool-use workloads.

Artificial Analysis@ArtificialAnlys

Artificial Analysis benchmarks Meta's Muse Glimmer, a 30B dense Apache 2.0 open-weights model. It scores 35 on the Intelligence Index and fits one GPU, but trails same-class models on agentic knowledge work and hallucination while leading on Tau3-Banking tool use.

Why it mattersMeta's first open-weights release since Llama 4 is Apache 2.0 and runs at full 128K context on one H100. Its weak GDPval-AA and 82% hallucination rate versus strong Tau3-Banking tool use show where it fits in agent stacks.

Artificial Analysis@ArtificialAnlys

Artificial Analysis benchmarks Meta's Muse Glimmer, a 30B dense Apache 2.0 open-weights model scoring 35 on its index. It runs on a single GPU, but lags Qwen3.6 27B on GDPval-AA and shows an 82% hallucination rate, while leading its class on Tau3-Banking tool use.

Why it mattersA 30B Apache 2.0 model that fits on one H100 or a high-end MacBook scores 35 on the index, but trails peers on agentic knowledge work and hallucinates heavily, which helps decide whether to self-host it.

Claude@claudeai

Anthropic is keeping Claude Sonnet 5's June launch pricing permanently: $2 per million input tokens and $10 per million output tokens, rather than letting the introductory rate expire on August 31.

Why it mattersSonnet 5 stays at $2 per million input and $10 per million output tokens after August 31, so builders can budget agent workloads on this model without planning for a price increase.

Simply run 'browser.backend: browser-use' to get started. More info on the brows

Nous Research's Hermes Agent swaps twelve browser tools for one that runs model-written Python over the Chrome DevTools Protocol.

Why it mattersReplacing a dozen browser tool schemas with one script-writing tool cuts per-request schema overhead and per-click round trips. Nous reports 48-66% fewer tokens with no accuracy drop, a pattern agent builders can copy.

articleNousResearch
Nous Research@NousResearch

Nous Research adds a Browser Use mode to Hermes that swaps twelve browser tools for one, driven by Browser Use CLI 3.0. The agent writes scripts instead of making a tool call per click, and the team reports 48-66% fewer tokens with no accuracy drop.

Why it mattersReplacing many tool schemas and per-click calls with agent-written scripts cuts token use by 48-66% in the team's tests. Browser agent builders can consider the same pattern to reduce context overhead.

Vercel Sandbox now runs on Vercel Managed Images

Vercel Sandbox moves onto open-source Managed Images.

Why it mattersAgent sandboxes now boot with coding agents already installed rather than installing at start, and you pick explicitly between auto-patched rolling tags and digest-pinned immutability. The old runtime property is deprecated.

articleAndy Waller

Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

A frontier checkpoint is around 500GB, so shipping one cross-region kills fast weight updates.

Why it mattersExplains why fewer than 1% of served weights actually change between RL versions, and how that fact lets rollout fleets live outside the trainer's cluster entirely.

videoAI Engineer

We asked an unreleased research version of Claude to take a stab at the Riemann

Anthropic reports an unreleased Claude raised the proven lower bound for zeta zeros on the critical line from 41.6% to 67.2%.

Why it mattersA research Claude coordinating about 60 subagents produced a result in analytic number theory that was checked by human experts and by Lean, concrete evidence of how far agentic research workflows reach.

articleAnthropic

Advanced capabilities require strong safeguards. That’s why access is limited to

Advanced capabilities require strong safeguards.

Why it mattersA new frontier cyber model exists, and access is restricted to approved defenders — that determines whether you can build on it at all.

OpenRouter@OpenRouter

OpenRouter's upgraded Auto Router classifies prompts into about 30 task types and ranks models by real community spend share over the prior 7 days, filtered by cost tier. It keeps multi-turn chats on one model to avoid cache rebuilds, with no extra fee.

Why it mattersYou can send openrouter/auto with a cost_tier and let routing follow which models the market currently spends on for each task type, at no added fee. Benchmark claims are OpenRouter's own and not independently verified.

Ant Ling@AntLingAGI

Ling-3.0-tiny open weights are out in BF16, FP8 and INT4. Reported results include Artificial Analysis Intelligence 25, Agentic 16, GPQA Diamond 73.40 and IMO-AnswerBench 71.03. Architecture is a 3:1 KDA-MLA hybrid with 128 experts, 8 routed and 1 shared. Local throughput figures are given for DGX S

Why it mattersA 1.3B-active open model with reported local speeds (about 100 tok/s on DGX Spark FP8) lets you run private, zero-API-cost agent tasks on a laptop or small box. Its Agentic Index of 16 suggests modest agent capability.

Meta Open Source Is BACK – Muse Glimmer First Test!

A hands-on test of Meta's Muse Glimmer across 3D CAD modelling, frontend design, creative writing, drum-kit simulation, multimodal coding and flight combat simulation.

Why it mattersFirst-hand evidence of how a new open-weight local model actually behaves on agentic coding tasks, before anyone commits to running it.

videoBijan Bowen

Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS

NVIDIA's Magpie multilingual TTS is a 364M-parameter open-weights model spanning 12 languages — Arabic, Korean and Brazilian Portuguese are new.

Why it mattersOpen weights at 364M put the TTS leg of a voice agent on infrastructure you control, so latency tuning, pronunciation customization and data residency stay yours rather than an integrated speech vendor's.

articlehuggingface.co

An index of the vibe-coding frontier. Corrections welcome.