Cua Releases Cua-Bench-S1 and Two Open Decision-Model Checkpoints
- Source
- x.com
- Date
1/ Introducing Cua-Bench-S1, a benchmark for decision models built for computer use, including Jev. We're releasing the first generation of Cua-S1 models, with two checkpoints: Cua-S1-Nano-0.1 and Cua-S1-4B-0.1 Cua-S1-Nano-0.1: https://t.co/ep6R2H4ahL Cua-S1-4B-0.1: https://t.co/p50GSsrXiP Repo:
2/ Here is what that looks like in practice. The clip shows a model making one chess move and working through a form task, each ending on a PASS screen.
3/ In every Cua-Bench-S1 task, the decision model reads one state and picks one option from a fixed list of element-and-action candidates. It decides once with no planning loop nor retries.
4/ Four task families share that format: interface actions, safety decisions about when not to act, game control, and held-out probes. The probes are chess, Doom, and JevBench, an external benchmark we use to see how a model handles tasks it was not trained for.
Context
Cua introduced Cua-Bench-S1, a for decision models built for computer use. In every task the model reads one state and picks one option from a fixed list of element-and-action candidates, deciding once with no planning loop or retries. Four task families share that format: interface actions, safety decisions about when not to act, game control, and held-out probes (chess, Doom and JevBench, an external benchmark Cua uses to see how a model handles tasks it was not trained for).
Cua also released two open research checkpoints: Cua-S1-Nano-0.1, an option- model of roughly 855K parameters trained from scratch, and Cua-S1-4B-0.1, a on Qwen 3.5 4B that reads text and a screenshot. Cua calls this gen 0, research artifacts and recipes, and notes that Cua-Bench-S1 scores offline datasets while Cua-Bench is a separate live-VM .
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
- LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Checking sign-in…
Loading comments…