Intel / article

Fantastic Adaptive Taxonomies and How to Use Them

Source: arxiv.org
Author: Mert Cemri, Andrei Cojocaru, Melissa Pan, Shu Liu, Shubham Agarwal, Alexander Krentsel, Jay Tang, Kannan Ramchandran, Joseph E. Gonzalez, Matei Zaharia, Alex Dimakis, Ion Stoica
Date: Published Jul 30, 2026

Context

An 's past mistakes become easier to reuse when they have consistent names. AdaMAST turns execution records into a compact classification of recurring failures, deriving the labels and their evidence patterns from the target system's own behavior. Those labels then supply feedback for subsequent attempts without changing the model's weights. In one reported comparison, this feedback raised SWE-agent's resolution rate on Mini from 60% with free-text reflection to 70%. The mechanism is to preserve recurring lessons across otherwise lengthy, task-specific records, giving the workflow a more consistent account of what went wrong. Research abstract.

Key takeaways · AI-distilled

AdaMAST reads an agent system's own traces and induces a failure taxonomy: named failure codes organized on three fixed axes — system-level, role-specific, domain-specific. No code is hand-authored and no trace is human-annotated.
Coded failure feedback beats free-text self-reflection at runtime: SWE-agent goes from 60% to 70% on SWE-bench Verified Mini, and Claude Code from 64.0% to 70.7% when the taxonomy is supplied as a runtime .
A verifier built on the induced codes picks among five candidate trajectories and lifts Terminal-Bench 2.0 accuracy 8-15 points over a single attempt, so the vocabulary works as a selection signal, not just a post-hoc diagnosis.
Taxonomies induced for different domains share few codes, so this is not portable boilerplate — each system needs its own. The compressed code set still matched expert failure annotations more closely than a hand-crafted reference vocabulary.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.

Key quotes

“We argue that an agent system should instead maintain an explicit representation of how it fails, induced from its own behavior and reusable wherever failure feedback is needed.”

“At runtime, taxonomy feedback raises SWE-agent's resolution on SWE-bench Verified Mini from 60% with free-text reflection to 70%, and improves Claude Code from 64.0% to 70.7% as a runtime skill.”

“In trajectory selection, AdaMAST-Judge, a verifier built on the induced codes, improves best-of-5 accuracy on Terminal-Bench 2.0 by 8-15 points over Pass@1.”

“Adaptive failure taxonomies close the loop between the traces agents produce and the procedures that improve them.”

Read the source arxiv.org