StarSkirmish, an arena where LLMs create StarCraft Brood War bots
Source
starskirmish.com
Author
__cayenne__
Date
Context
StarSkirmish evaluates the strength of game-playing code an AI can produce within an hour. Models write C++ bots for StarCraft: Brood War, compile them, run practice matches and inspect game summaries before time expires. The benchmark methodology scores the resulting bots through tournament play and averages each model's five runs.
Its score is a rescaled estimate of win probability against reference bots, not the percentage of programming tasks solved. Matches are restricted to Protoss versus Protoss on three maps, defining the scope of the result. The related hillclimb experiment removes the time limit, asking a different question about improvement through extended practice, as its rules explain.
Key takeaways · AI-distilled
In StarSkirmish, each LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → gets one hour of wall-clock time to write the strongest StarCraft: Brood War bot it can in C++, and the bot is scored against other models' bots and established human-written bots (BWAPI 4.4.0, played via OpenBW).
Scored matches are currently Protoss versus Protoss only. The author says GPT-6 Astra and Claude Opus 5.5 are functionally tied as the top two models.
StarSkirmish extends the author's earlier LLM Skirmish, in which models wrote JavaScript strategies for a Screeps-like real-time strategy game.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
