ExplorationBench: Feedback and Self-Designed Probes Drive Agent Discovery

ExplorationBench asks whether an AI system can learn rules nobody has told it by running its own experiments, and its answer is yes for the strongest systems, though not reliably. Researchers from Tencent's Hunyuan team, Fudan University and Tsinghua University built two small worlds whose rules can be executed, so every answer is checked exactly, and whose rules deliberately contradict familiar ones, so recalling training data does not help; Tencent Hy announced the work on X. AlienCode is a little programming language with 31 hidden changes: integer literals are silently XOR-ed with 27, so EMIT(100) prints 127, and PLUCK counts positions from one although the manual says zero. AlienLogic is a formal proof system with 24 altered inference rules, and some of its theorems cannot be proved, so the only correct answer is to decline. Each system starts from a manual that is wrong about these changes and a fixed set of worked examples, then spends four rounds submitting programs or proofs and reading the results, which come back without explanation. After each round a copy of the conversation with tools switched off states the rules it believes and answers 70 held-out tasks, graded by an interpreter or a proof checker rather than an AI judge, and the copy is then thrown away so testing never feeds the exploration. The same group's CL-bench put the knowledge a model needed directly in its prompt; here the model has to collect that evidence itself.

The gains came from evidence, not extra thinking. Before exploring, no AlienCode trajectory scored above 15.7%; after four rounds the best reached 89.0%, and 7 of 10 systems passed 60%. Giving systems the same number of model turns with no environment feedback left them at 0.5–11.0%, although that comparison is uneven: headline scores take the best of three independent runs per system, while each control was run once. A sharper test asked who should design the experiments. Because the world is deterministic, replaying a system's own best sequence of probes hands it the same evidence, yet in AlienCode the median score fell from 67.4% when systems chose their probes to 41.2% under replay, and 9 of 10 systems did worse, while a fixed set of probes shared by every system yielded a median of 5.7%. The authors read this as the value of designing the experiment rather than just receiving its results. The effect did not carry over to AlienLogic, where self-chosen and replayed proofs tied, and a replay is itself a single run that can vary widely: three replays of one system's probes ended at 47.1%, 10.0% and 46.7%.

Finding a rule and using it also came apart. When a system's final rule report stated every rule a task needed, it still solved that task only 73.4% of the time; in one case a system knew the index shift but applied the wrong number encoding inside a function, so its indexes ran out of range. The two worlds even point in opposite directions. In AlienLogic, simply being handed the full rule set scored 96–100%, better than any system's exploration, while in AlienCode 7 of 10 systems' best runs beat being told the rules at the start. Being told the rules after exploring also beat being told them beforehand in 26 of 30 AlienCode runs, which the authors read as exploration supplying practice in using the rules as well as the rules themselves. Results also swing between attempts: runs of one system under one budget ended up to 73.3 points apart in AlienCode, while answering the same questions again at the same point varied with a standard deviation of at most 4.7 points, and a system's rank in one world barely predicted its rank in the other. The paper treats these as measurements of acquiring unfamiliar rules in two controlled synthetic worlds over four rounds, not of real scientific discovery, and notes that three runs per system are too few to estimate that variation precisely.