Vibeleaderboard
Index — Latest Intelligence

Intel

Page 60

How to Train a Real-World Silicon Concierge? Internalizing Complex Business Workflow to Only OneModel

An industrial team argues the Router/Retriever/Planner/Executor pipeline degenerates into ad-hoc patches with cascading errors and high latency.

Why it mattersA production-scale argument against modular agent orchestration: internalizing workflow logic into the model removes the cascading errors and latency that come from patching a chain of components.

articleChang Liu, Chaoyang Ning, Dayi Jiang, Enrui Gu, Fang Ran, Hongyan Xue, Huaqing Li, Hui Cai, Jia Liu, Jiang-Ming Yang, Jianshe Li, Jiawei Luo, Jin Zhou, Leshen Zhu, Lihui Chen, Liying Ma, Lyuxin Xue, Mengjian Ji, Ruijia Xu, Wei Ren, Wei Wu, Xiaoling Qu, Xiaoyun Feng, Xin Zhang, Xixie Zhou, Xuanwei Hu, Yan Chen, Yichao Wang, Yongqi Tong, Yu Liu, Yuhong Zhou, Zemin Sun, Zhenwen Xu, Zhiling Liu, Zifan Wang

Testing and Evaluation of Agentic AI Systems In Military Command and Control

An assurance case needs claims, evidence, and an argument connecting them.

Why it mattersEstablished test-and-evaluation methods assume a system is specifiable, stable, composable and supervisable — agentic behavior weakens all four, so a passing evaluation does not license inferences about deployed behavior.

articleUlysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan

An Extensive Empirical Study on Code Translation Technique

A large comparison of learning-based methods, LLM-based methods and general-purpose LLMs on multilingual method-level and class-level code translation, scored on executable correctness rather than similarity.

Why it mattersIf you are migrating code with an LLM, this maps where it actually holds up — and shows that similarity scores are a bad proxy for whether the translated code runs.

articleRuihang Fan, Jiajun Jiang, Xinpeng Wang, Jiateng Fu, Fengjie Li, Jiasi Shen

BC-Bench: Evaluating Agentic Engineering in a Domain-Specific Language for ERP

Frontier coding agents are strong on general benchmarks and untested on enterprise DSLs.

Why it mattersAgentic coding gets measured somewhere other than Python: 101 real tasks in Dynamics 365's AL DSL, with multi-run scoring that separates model differences from run-to-run nondeterminism.

articleHaoran Sun, Klaus Marius Hansen

Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors

103 participants, sorted by a proficiency assessment, debugged Python against three kinds of error message.

Why it mattersLLM-rewritten error messages feel clearer and less taxing than raw interpreter output, but the measured debugging performance — fix rate, attempts, time-to-fix — does not follow the perceived improvement.

articleAlexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju

Vercel Sandbox is now globally available

Vercel Sandbox now runs in four regions — iad1, sfo1, cle1 and cdg1 — with iad1 still the default.

Why it mattersIf you run agent code execution on Vercel, you can now place sandboxes near your database and storage instead of paying a transatlantic round trip, and configure failover — but snapshots cannot move between regions.

articleMarc Codina Segura

When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha

Two benchmarks — 64 validated Gen Alpha mental-health expressions and 75 paired multi-turn conversations.

Why it mattersDecoding slang is not safety: models that parse adolescent phrasing still miss the risk it encodes, and the paired Standard/Gen Alpha conversations give a reusable way to test that gap in your own product.

articleManisha Mehta, Virendra Mehta

Temporal Validity on Real Software Histories: Eliminating Stale-Fact Errors in Code-Assistant Memory over GitHub Fixes

RAG has no clock. When a function is renamed or a dependency bumped, the stale and current statements retrieve at near-identical similarity and the assistant serves the old one.

Why it mattersRetrieval-based code assistants serve stale facts about a third of the time when a value changes mid-session — a renamed function, a moved endpoint — and an LLM reranker doesn't fix it; supersession-aware memory does.

articleNeeraj Yadav

When Retrieval Fails Before It Begins: Structurally Indirect Prerequisite Eviction as a Retention Failure in Agentic Memory

Agentic memory research usually tunes retrieval and assumes the needed evidence survived eviction.

Why it mattersIf your agent's memory runs under a token budget, the block it needs can be evicted before retrieval ever runs — tracking what a chunk depends on, not just how well it matches the query, recovers it.

articleMinkyu Song

Generation of Web Apps with Agentic IDEs: An Empirical Assessment

Three agentic IDEs — Copilot, Cursor, Windsurf — pointed at five full-stack web apps built from scratch.

Why it mattersAgentic IDEs handle established patterns like CRUD and auth reliably but produce significantly more errors on less common distributed architectures such as task queues — scope what you hand an agent accordingly.

articleManuel Marceca, Maria Teresa Rossi, Leonardo Mariani

Nexus: Depth-Adaptive KV-Cache Splicing and Retrieval-Decoupled Tool Routing for Agentic LLMs on Unified Memory

MCP agents re-send every tool schema each turn, so prefill dominates time-to-first-token as the registry grows.

Why it mattersA concrete answer to tool-registry bloat: route tools by retrieval over a compact signature instead of re-prefilling schemas, and treat KV-cache splicing as bounded by RoPE phase drift rather than by effort.

articleMustafa Arslan

Vibe Coding: Practice, Performance, Productivity, and Risk -A State-of-the-Art Review

Seventeen months of vibe-coding evidence pulled together across software engineering, HCI, labor economics, security and governance.

Why it mattersThe productivity evidence for AI-assisted coding openly contradicts itself: +26% tasks per week in field experiments, a 19% slowdown in randomized trials, and code-review time up 441% in team telemetry.

articleDominik L. Michels, Mutaz Abu Ghazaleh, Francois Lazzari, Nabil Kassem, Jonathan Klein

Who Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational Bias

Models that pass behavioral bias evaluations may still encode the bias.

Why it mattersClean behavioral bias scores do not mean the representation is clean — intervening on a model's internal estimate of user expertise changes its output, so output-only eval suites can miss the disparity entirely.

articleKeren Fuentes, Aaron Mueller

Representation Affects Retrieval: A Case Study of Skill Discovery and Routing in a Multimodal Agent Harness

A production multimodal video-agent harness details how it exposes skills to its planner.

Why it mattersHow a skill is represented in the system prompt, not just which skills exist, decides what the planner picks — and in-context selection is the small-N counterpart to embedding-based retrieval at scale.

articleKevin Dela Rosa

Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality

A token-level analysis of 132,000 prompt variants finds that prompts scoring higher on a task are also the ones stable under perturbation.

Why it mattersPrompt robustness is not luck: naming domain terms precisely and giving explicit action directives narrows the model's interpretive space, and the prompts that score highest also vary least across rewordings.

articleQipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu

AgentX: an open benchmark for million-token agentic inference

SemiAnalysis open-sourced AgentX 1.0, an Apache-2.0 benchmark measuring inference on real multi-turn agentic coding traffic at 1M context rather than fixed prefill/decode.

Why it mattersHardware and serving comparisons built on single-turn 8k/1k traffic mislead for agentic workloads; this one models KV-cache reuse and sub-agent bursts, and the dataset and dashboard are open.

articleSemiAnalysis

Why model routing must be in the harness

Model prices among agent-capable models span fiftyfold, and one task can burn hundreds of calls — so where the routing decision lives matters.

Why it mattersRouting models inside the agent harness instead of at the gateway cut aggregate inference cost 58% and median turn latency from 81s to 49s in production, with task-completion and failure measures holding steady.

articlefactory.ai

Fences, not Sandboxes

Yegge describes running 50-60 agents on 21 Claude Max accounts, with Fable officers managing Sol and Opus fleets.

Why it mattersDetails a real large-scale agent organization: long-lived Fable officers supervising Sol and Opus fleets, with costs disclosed, and argues for governing agents with rules rather than containment.

articleyegge.ai

How To Use Playwright Mcp With A Cloud Browser

A step-by-step on handing an agent a real browser over MCP.

Why it mattersGives an agent a browser it can click and fill rather than only page text, with runnable config for both the local dev path and a hosted browser that survives bot checks.

articleBrowserbase

camsnap: a CLI that hands an agent live camera frames

A Go CLI that saves cameras by name, then captures snapshots and short clips from RTSP/ONVIF devices and local webcams, discovers ONVIF hosts on the network, runs health checks.

Why it mattersOne command turns an RTSP or USB camera into a JPEG an agent can read, with motion triggers that shell out — no bespoke ffmpeg plumbing to give an agent eyes on the physical world.

articlePeter Steinberger 🦞
SemiAnalysis@SemiAnalysis_

SemiAnalysis metered the $200/month plans against API list prices: roughly $8,000/month of tokens from Anthropic and up to $14,000 from OpenAI if you exhaust the weekly limits on long-horizon work. The test dates to June, so treat the ceilings as a snapshot rather than current policy.

Why it mattersQuantifies the gap between subscription pricing and API rates for heavy agent use — roughly $8,000/month of Claude tokens and up to $14,000 from OpenAI — which changes whether you build on a plan or on the API.

Anthropic’s best AI model struggles to attract users as cheaper tools thrive

FT-sourced figures — Anthropic at $65bn annualized in July, OpenAI past $40bn — paired with Ramp's index of card spend across 70,000 companies.

Why it mattersRamp's billing data gives an outside view of which models companies actually pay for: Opus 4.8 at 28% of Anthropic spend while Opus 5 and Fable 5 sit near 3-8%, suggesting price is steering adoption more than capability.

Dependabot now waits three days before non-security version update pull…

GitHub has made a three-day cooldown the default for Dependabot's non-security version update pull requests, on the reasoning that maintainers and researchers usually surface problems in a fresh release within that window.

Why it mattersIf you rely on Dependabot, routine version bumps now arrive three days later by default — a deliberate trade of update latency for a window in which a compromised release can be found before it reaches your code.

articleGitHub

QuantBench: Benchmarking AI Methods for Quantitative Investment

QuantBench is an industrial-grade benchmarking platform for AI methods in quantitative investment, standardizing the full pipeline across temporal models, graph neural networks and end-to-end reinforcement learning.

Why it mattersFirst standardized benchmarking platform for AI methods in quant investment — the kind of niche benchmark the exhaustive benchmarks coverage exists to surface.

articleSaizhuo Wang et al.

How This 5x Founder Runs His Startup Solo With AI Agents (OpenClaw, Codex, Devin) | Ryan Carson

Peter Yang sits down with five-time founder Ryan Carson to see how he runs his startup solo with AI agents, splitting real day-to-day work across OpenClaw, Codex.

Why it mattersA repeat founder replacing an early team with named agent tools is a concrete, current datapoint on how far agentic workflows go in production.

videoPeter Yang

The Exact AI Skills This Solo Founder Uses to Build 5 Apps at Once | Josh Pigford

Peter Yang interviews Josh Pigford about the exact AI skills he uses to build five apps at once as a solo founder, from agent-driven coding workflows to the daily habits that keep every product shipping.

Why it mattersA named solo founder showing the actual AI workflow behind shipping five products at once is exactly the practitioner playbook content the Intel feed exists to surface.

videoPeter Yang

The Agent Behind the Curtain: Building the Oz Cloud Agent Platform — Safia Abdalla, Warp

Warp's repo went from roughly 20,000 to over 60,000 stars after open sourcing, with contributions arriving faster than humans could read them.

Why it mattersA working pattern for surviving a contribution flood.

videoAI Engineer

llm 0.33

Simon Willison released llm 0.33.

Why it mattersTemplate composition turns a model plus its options into something you name once and reuse everywhere.

SemiAnalysis@SemiAnalysis_

SemiAnalysis proposes tokens per second per provisioned megawatt as an inference metric and works a B300 example on DeepSeek V4: roughly 7,400 output tokens per second per megawatt. Human speech, at 3.3 tokens a second on a 20 watt brain, comes out about 22 times more efficient.

Why it mattersPower, not GPU count, is the binding constraint on serving capacity, so a metric denominated in megawatts is the one capacity planning will use.

Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo

Sebastian Fox broke three leading production ambient scribes in one afternoon, then shows why the obvious fix fails: his best checker.

Why it mattersA verifier that diffs source against output catches only the easy half; deciding which omission matters is tacit and never written down, so rubrics miss serious errors.

videoAI Engineer

GitX

A portable git workflow skill for AI coding agents that turns a messy working tree into logical conventional commits.

Why it mattersAgents produce commits nobody can reconstruct later, and moving the git discipline into a skill is a direct fix rather than a reminder.

repoMusoyanGrigor
repoMusoyanGrigor

I turned Apple-style scroll-video websites into a reusable AI skill

An agent skill that takes a video path and returns a scroll-scrubbed landing page.

Why it mattersPackages a fiddly scroll-video build — frame extraction, encoding trade-offs, progressive loading — into one command an agent runs, with a confirmation step before it rewrites assets.

namo_complete

A Bash autocomplete that waits until you pause, then prints a dim suggested command on the line below, which Alt-O accepts into your prompt for editing.

Why it mattersSuggestions that stay out of the way until you pause are a gentler shape for AI in the shell than a chat box.

repodavidwbrwn
repoParthAkholkar

Stash, turn your Instagram saves into notes Claude finds on its own

A local pipeline that pulls saved Instagram posts, transcribes reels with Whisper on Groq, reads static and carousel content with a VLM.

Why it mattersA working pattern for turning a capture backlog into agent-readable memory: transcribe at ingest, store as searchable markdown, and expose it over MCP so it surfaces unprompted rather than waiting to be queried.

Agent Frameworks Considered Harmful — Rémi Louf, .txt

Rémi Louf's agents duplicated posts, dropped a voice note, and degraded after unversioned prompt edits, so he replaced the framework with a runtime.

Why it mattersHashing each prompt component turns a prompt into a list of hashes you can diff between runs and rebuild against a different model — visibility a live chat session can't give you once compaction and hidden reasoning are involved.

videoAI Engineer

More than just code review

Simon Willison argues that the core skill with coding agents is instructing them precisely and then verifying the result.

Why it mattersIt puts a name to the skill teams need to train for as agent output outpaces the capacity to read it.

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

DigitalOcean's Archana Kamath and Tyler Gillam argue there is no best model, only the right one per request, and show a router driven by declared preferences.

Why it mattersPer-request routing held quality near a single premium model (90% vs 95%) while cutting a coding session from 25 cents to 8, and the routing model and proxy are open sourced.

videoAI Engineer

Can I Run This Model?

A VRAM calculator for local language, image and video models that counts the text encoder and runtime overhead alongside the weights, and shows every formula it uses.

Why it mattersThe gap between a model's weight size and what it actually needs on the card is where most local deployments fail, and this shows the arithmetic.

articleAadi1277

What If Your Chip Design Team Moved Like a Single Body? — Abduallah Mohamed, AIDAChip

A chip-design team watched their agent route around every tool-level block — bash, then sed, then cat — and moved enforcement down to the substrate.

Why it mattersTool-by-tool denial fails against a capable agent; enforce at the substrate. The human-approved intent graph is a reusable pattern for keeping multiple specialized agents consistent about ground truth.

videoAI Engineer

Figranium – Build browser tasks visually, execute via API (Dockerized)

A self-hosted control plane that turns visually built browser flows into authenticated API endpoints.

Why it mattersGives agent and n8n pipelines deterministic browser steps behind an API endpoint, with proxy rotation and captures, without usage credits, rate caps, or third-party hosting of the scraped data.

repoasernasr

FinOps for AI Agents: Who Spent All the Tokens? — Tisha Chawla & Susheem Koul, Microsoft

Microsoft engineers put a control plane between application code and model spend.

Why it mattersThrottling controls cost by killing runs; steering cut spend ~78% while lifting completion from 67% to 96%. Preview mode lets you evaluate policies against a production agent before enforcing them.

videoAI Engineer

Knowl

Persistent memory for Claude Code, Cursor and Codex over MCP or a CLI, built around supersession: when a fact changes, the old one is retired instead of competing with the new one.

Why it mattersEvery long lived agent project accumulates contradictory context, and retiring superseded facts is a real mechanism rather than another instruction to keep the file tidy.

repodat999zx

Conveyor

An open source work queue that makes coding agents plan against confirmed requirements and design documents before implementing.

Why it mattersTeams running several coding agents hit review capacity long before they hit code capacity, and this is a specific process answer: approve intent first, then trace every change back to it.

repokidustiliksew

Give the Agent a Budget, Not a Token — Sachin Malhotra, Anthropic

An Anthropic engineer's agent wiped ~200 workloads in 90 seconds using his own token.

Why it mattersScoped tokens are booleans and don't bound blast radius. Budgeting agent authority along magnitude, rate, reversibility and visibility is a design you can apply to any agent that holds write access.

videoAI Engineer

dsh-edge

Runs the DeepSeek Harness web coding agent on Cloudflare Workers with a durable workspace that persists conversations and files between browser sessions.

Why it mattersA coding agent that lives in a durable edge runtime rather than on a laptop is a working answer to where a personal agent should run.

repopawaca

pisesh

A keyboard driven terminal browser for pi coding agent sessions.

Why it mattersSession sprawl is the quiet tax on long running agent work, and this is a small tool aimed squarely at it.

repoblue-b

VinvAI runs and observes your services to find bugs and perf issues

An open-source runtime evidence layer that runs your services locally, ties traces back to the source that produced them, hands that evidence to your coding agent, then re-runs the code and gates the fix on acceptance tests the agent never sees.

Why it mattersJudging agent fixes on what the code actually did, against tests the agent cannot see, is a concrete answer to agents that claim success without it.

articleitsAnshul

SkillWorks – every Claude Code skill, scored on whether it loads

A nightly-rebuilt index that reads every Claude Code skill, subagent, plugin and marketplace from source and scores each one from 0 to 100 on four disclosed components, starting with whether the listing loads at all.

Why it mattersAnyone installing Claude Code skills, subagents or plugins can check whether a listing actually loads before wiring it into their stack.

articlekyisaiah47

Plainsay – 12 MB Mac dictation with local Whisper and Parakeet

A free, MIT-licensed Mac dictation app in a 12 MB download that runs Whisper and Parakeet on the machine itself, placing text at the cursor with no account, no telemetry and offline recognition once the model is downloaded.

Why it mattersLocal Whisper and Parakeet in a 12 MB signed app is a practical way to dictate prompts to coding agents without sending audio anywhere.

repoconrader

How Claude Watermarks AI-Generated Text

A recorded lecture with transcript that walks slide by slide through the mechanism behind Anthropic's text watermarking of Claude output, expanded from a short social post into roughly 50 slides and 48 minutes of detail.

Why it mattersWatermarking changes what teams can assume about detecting and attributing model output, and this is the clearest available explanation of the underlying mechanism.

articleSebastian Raschka, PhD

An index of the vibe-coding frontier. Corrections welcome.