Vibeleaderboard
Index / tool
Visit github.com
Category
AI Agents
Rank
No. 4142Tools index
Pricing
Open Source
Type
TOOL
GitHub
1 stars
Latest release
v0.3.0
Date

About

A test-time search harness that wraps a coding agent CLI, verified with Claude Code, in Monte Carlo Tree Search. Each node is an isolated git worktree with a forked agent session, and the working tree is untouched until the user applies a result. The README reports that on the tasks measured so far the search did not yet beat a single attempt.

At a glance

agent-mcts is a Python command-line harness that runs a coding agent many times on one task using Monte Carlo Tree Search. Each attempt gets its own git worktree and forked agent session, your tests score it, and the search spends more of its budget on promising branches. You then apply the best branch as a staged squash merge. By the author's own v0.3 benchmark, it does not yet beat a single agent run, so treat it as an experiment.

Good fit

Researchers and tinkerers studying test-time search over real software tasks, especially with Claude Code (Anthropic), the only agent verified so far. It suits a repository with a test suite that reports pass counts, since the tests are the scoring signal, and anyone who wants to tune UCT constants and selection settings rather than just get a fix.

Consider something else when

You want better answers from your agent today. On the author's two harder benchmark tasks, tests-only search matched the first attempt in 9 of 10 runs, and adding the LLM judge averaged a 0.1 point loss while taking about 23 minutes instead of 5. Also skip it if your project has no automated tests, or your agent is Codex CLI and you need a tested setup.

Start here

Run uv tool install agent-mcts, change into a project, then run agent-mcts run "<task>". It detects the agent and test command and shows a plan, such as 12 nodes and a $10.00 cap, before asking to proceed. Watch the live tree, then use agent-mcts show to inspect nodes and agent-mcts apply to stage the best branch for your own review and commit.

Cost and requirements

MIT-licensed and requires Python 3.11 or newer plus an installed agent CLI, whose model usage is what you pay for. Budgets are set per run with max_nodes and max_cost_usd. The --judge option used about five times the episodes and wall time in the author's benchmark. For uv-managed Python projects, set the value command to uv run pytest -q, because fresh worktrees do not inherit your virtualenv.

Limits to know

The project calls itself early (v0.3, classified pre-alpha). The Codex adapter has only been tested against a fake binary, and Kimi support is planned for v0.4. When two attempts both pass the visible tests, the judge picked the better one 38 times in 62, and in one run it swapped a 91% answer for a 68% one. Interrupted runs keep their partial tree and can be resumed.

Tags

mctscoding-agentclaude-codetest-time-searchpythoncli

Tech Stack

Python

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.