Vals RSI Index: can models build their successors? Muse Wonder Full ranks 9th
- Source
- vals.ai
- Author
- Vals AI
- Date

Context
Vals AI's RSI Index, named for recursive self-improvement, asks whether a model working as an autonomous can do the research that goes into building its successor (vals.ai). Each of its four tasks hands the model an open-ended AI R&D problem with a fixed compute and time budget: build a lossless compressor for the enwik8 text corpus, edit a starter GPT training program to get the lowest bits per byte on held-out data, improve a Gemini 3.5 Flash-Lite judge so its grades of Vibe Code Bench apps agree with human graders, and post-train a supplied Qwen checkpoint into a better finance agent. Every model runs at its highest reasoning effort inside an off-the-shelf coding agent, usually its maker's own (Muse models in Muse Code, Claude in Claude Code, GPT in Codex), loops until the budget runs out, and submits one artifact that Vals re-runs or retrains and grades. Scores sit on a log scale where 0 is the task's starting baseline, 0.5 matches a chosen reference result and 1 is the theoretical best; the index is the average of the four. Vals says those references mix historical research results, human annotator agreement and a frontier model, so 0.5 is not a uniform measure of the best human work. This is the index Vals introduced in September (earlier coverage).
On that scale, Meta's Muse Wonder Full (launch coverage) ranks ninth at 24.4%, which Vals says is 4.8 points above Muse Spark 1.3 Max (19.64) and one point above Claude Fable 5 (Vals thread). The average hides very uneven tasks. Its best was LM training, where its program reached 0.8920 bits per byte against the 0.85 published human reference, a task score of 0.44, with only four models ahead of it. In the judge-building task its agreed with human graders 78.05% of the time (score 0.19), against 93.6% for the best-agreeing pair of human annotators. Compression was mid-pack at 0.35. In post-training its submitted checkpoint scored 7.01% against the untouched parent model's 35.38%, which counts as 0, the same zero most models received there.
Read against the whole board, a quarter of the way up the scale is still well short of the references. Across all 25 models, only one task result passed its reference: Claude Opus 5.5's 0.8308 bits per byte in LM training; Vals says the references remain ahead in compression, judge-building and post-training, and the second-ranked model, Claude Fable 5.1, scores 0.36 overall. Vals also cautions that these are single runs on narrowly scoped tasks with fixed budgets and off-the-shelf harnesses, and that more compute, better orchestration or research tooling could change the picture, so a one-point gap such as Wonder Full's over Fable 5 is a thin margin to read much into.
- Vals scores four autonomous research tasks, compression, LM training, judge-harness engineering and post-training, on a log scale where 0 is the starting baseline and 0.5 a published, human or frontier reference; the index averages them.
- In LM training Claude Opus 5.5 reached 0.8308 bits per byte, beating the published 0.85 reference with a recipe trained 30 minutes on one H100 per seed, though Vals says its protocol is not a matched reproduction of the original paper.
- Despite having an H100, the leading enwik8 compressors used adaptive mixing rather than neural compression, and none reached the 1.2435 BPB nncp reference; Claude Opus 5.5 led at 1.3120.
- Experiment volume did not predict quality: GPT-5.6 Sol logged 114 LM-training trials yet trailed most models, while Claude Fable 5.1 ran 47 and placed second; Vals says idea choice and final-candidate checks mattered more.
- Only six of 25 agents beat the 35.38% parent in post-training; Gemini 3.8 Flash fell to 3.54% after picking a checkpoint on simulated tool runs, while Claude Fable 5.1 rejected its adapters and gained via prompt guidance alone.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…






