PokerTools Arena

At a glance

PokerTools Arena seats language models at a no-limit Texas Hold’em table and exposes their decisions, errors and replay history. For developers investigating model behaviour, the useful part is being able to inspect a choice and its legal alternatives. It merits a trial if you can configure API access and interpret the results carefully. A tournament winner or a ten-spot sanity score does not establish general reasoning ability or poker strength.

Good fit

Comparing how configured models handle the same decision format, or finding whether failures come from response formatting, timeouts or action selection. The engine supplies legal choices and a player-specific view; the spectator can see more than a model receives. TypeSafe AI’s Jev and OpenAI-compatible endpoints use different adapters. Hierarchical decisions choose an action family first, then a size when needed, making those two mistakes distinguishable.

Consider something else when

You want a poker solver, a validated model ranking, or a ready-made match without supplying model access. Strategy mode supplies a deterministic hand classification; Raw cognition mode leaves that inference to the model. Keep their results separate. The repository’s published diagnostic sample is small, omits some comparison dimensions and explicitly attaches no real tournament A/B run to that release.

Start here

Open the hosted app, then Settings → Connections to choose an endpoint and its credentials. Click a table seat to enter a model and supported protocol. Before seating opponents, use Tests to inspect the ten fixed scenarios and run one configured model within your provider budget. Check errors as well as expected-action matches before starting a two-seat tournament. The setup, seat and Tests dialogs were inspected here; no paid request or model match was run.

Cost and requirements

The arena code is MIT licensed. The hosted browser app avoids local installation; the optional local launcher/tooling requires Node.js 24 or later. Your provider or local model supplies inference. Tests use real model calls, and an aggressive hierarchical move can require both family and sizing calls, plus retries. A connection’s Test result does not establish that its decision protocol works. Browser access also depends on the endpoint allowing cross-origin requests.

Limits to know

Game orchestration runs in the browser, but selected providers receive decision context and API authentication. Source code excludes API keys and custom headers from saved setup; tournament state and reported reasoning can still be stored locally or exported, so inspect logs before sharing. Errors can trigger automatic fallback actions, which must not be mistaken for model choices. Fixed-state comparisons and live tournaments answer different questions because each move changes later states. No provider compatibility, masking, gameplay or comparative-quality trial was performed for this guide.

Related work

• TypeSafe AI (context)

Still unclear

• Whether your exact endpoint/model/protocol combination completes decisions reliably within the configured clock and budget; no authenticated inference was tested.