When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop

Running a model's own code to check its training answers cannot correct a misunderstanding that the model has already written into that code, according to a study by three AltSlate Labs researchers. In the self-play loop they tested, one small model writes a short Python task, an answer key is produced, the same model attempts each task eight times, and it trains on its attempts that match the key. Loops like this are often called grounded when the key comes from executing code. The authors fine-tuned Qwen3-4B-Base to misread two task phrasings, treating "at most n" as strictly below n and "the k-th smallest" as counting from zero, then varied where the key came from. When it came from running the model's own program, the program carried the same "at most" misreading on every task they audited, and the share of correct answers never rose above 3 percent. When it came from a program written by the original model, before the misreading was planted, the share rose to between 23 and 58 percent after three rounds in all four runs.

A correct key was not enough on its own. The loop keeps a task for training only if the model solves it between one and seven times out of eight, so a belief held on nearly every attempt produces almost no matching answers to learn from. In one six-round run, the model still counted from zero on 96 percent of answers, down from 100 percent. Where the correction did work, a different wrong rule came with it. After fixing "at most", the model increasingly counted the nearest value above the limit when the limit itself was missing from the list, giving too high a count on 18 to 38 percent of those answers by round 3. The model in these runs had almost never written such tasks (4 of 10,924), so every correcting example was consistent with both the true rule and the new wrong one. The authors offer this as one of three explanations they cannot separate. Asked directly for those cases, it wrote none in 2,061 checkable tasks, and both it and the unmodified model often rewrote the request into the task wording instead, for example "the cap itself does not count".

The results come from one four-billion-parameter model, three to six rounds and two random seeds, with deliberately planted errors and a clean key writer that the authors call an upper bound, because in practice nobody knows in advance which model lacks a given error. They did not test whether adding the missing cases removes the new rule. The narrower point holds within those limits: execution checks what code does, not what a task means, and a fix should be scored on nearby cases where the old belief was right. In one six-round run, the fix looked nearly complete on the cases it targeted, with 88 percent of answers correct, while the model's single best answer was right on only 23 of 50 nearby cases, down from 37.