NeuroGolf fourth-place solution: agent loops, shared idioms, and runtime budgets
- Source
- kaggle.com
- Author
- hongan, MaxChen303, Maciej Sypetkowski, Natalia Jaśkowska, mrmldjr
- Date

Context
In the fourth-place solution to Kaggle's NeuroGolf competition, a network with very few parameters could still be too slow to run. The team's writeup says some operators, notably Einsum, could use only a few parameters yet take hours to run all examples on a single CPU. Runtime therefore became a constraint separate from a model's score.
The team's agents each worked on one task at a time and repeatedly improved it, while a registry retained multiple versions of each task's model by hash, with their scores, local times, and leaderboard status (unknown, correct, or broken). For the final submission, the team approximately maximized the total score of the selected models under a budget defined by their summed evaluation time. Keeping alternatives mattered: the writeup says choosing every task's highest-scoring model could produce a submission too slow to finish. It is a concrete example of optimizing an entire workload rather than each component independently.
- Store candidate versions by hash with score, runtime, and external-validation status; the locally highest-scoring version is not always the best submission choice.
- Share reusable optimization idioms while supplying each task only its relevant comments and a compact summary of other feedback.
- Validate unknown candidates through grouped probes, isolate failures, and optimize the final selection under a total runtime constraint.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Checking sign-in…
Loading comments…

