Brood War Bench

Brood War Bench has AI agents play the real-time strategy game StarCraft: Brood War, and its author reports that thinking time can undermine an agent in a game that keeps running. The author says agents were often drawn into spending dozens of seconds deciding how to answer a single Probe (a worker unit) sent across the map to attack, instead of doing anything else. The author also saw Codex agents split economy, unit production and army control among separate subagents that communicated little, so the army agent sent each new unit into attack one at a time instead of waiting to assemble a force. Those are the author's observations from watching games, not a tested explanation of the results. The report's overall finding is that none of the models played beyond a beginner level.