Vibeleaderboard
← All Intel
Intel / post

Vals AI launches CUA-Bench to test real-time computer use

Source
x.com
Date
Vals AI@ValsAI

What tasks are left that humans find easy but today's models still find hard? Two such tasks are computer use and games. We’re launching CUA-Bench, a benchmark testing how well AI can use a keyboard and mouse across 6 games (3 kept private) in real time. To saturate it, models will need to output real-time actions and learn continuously from video, not just text.

Context

Vals AI launched CUA-Bench, a measuring how well AI models can operate a keyboard and mouse in real time across six games, three of which are kept private so models can't be trained on the exact test set. Vals AI frames the benchmark around a specific gap: computer use and games are tasks humans find easy but that current models still struggle with, and the company says saturating CUA-Bench will require models that output real-time actions and learn continuously from video, rather than models built primarily around processing text.

Terms in this piece · Glossary
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
More from Vals AI
Recommended reads
Comments

Checking sign-in…

Loading comments…