Intel / article

Qwen Intelligence launches three mobile agents and opens its benchmarks

Source: github.com
Author: Qwen
Date: Published Sep 23, 2026

Context

Qwen introduced Qwen Intelligence on September 23 with three phone agents: a Mobile Planner that breaks a request into steps and coordinates tools, a Mobile-Use Agent that carries out tasks through app APIs and falls back to operating the on-screen interface, and a Mobile Creative Agent that turns a sentence into images and other creations. Qwen called all three state of the art. For the planner and the Mobile-Use Agent, the scores behind that claim come from benchmarks whose authors overlap with the Alibaba teams that wrote the agents' reports, and in the planner's case the lead is a fraction of a point.

The Qwen-Planner-Agent report gives the planner, a 27-billion-parameter model paired with software that supplies memory, reusable procedures and helper agents, an Overall score of 77.05% on MobilePA-Bench. GPT-6 Astra scored 76.84% and Claude Opus 5 75.71%, so the first-place claim rests on a 0.21-point margin. MobilePA-Bench tests 1,705 requests against 212 simulated phone tools, and all 11 authors in the citation on its site also appear on the planner paper. That site's leaderboard, which says it uses revised success criteria, names GPT-6 Astra as the best Overall score at 76.87%, slightly different from the report's figure, and the text captured from the site does not show a score for Qwen's planner. The report's clearer advantage is cost: an estimated $2.41 per 1,000 tasks in output , against $3.06 to $67.76 for the other models it priced, with input tokens, tool charges and device time excluded.

The Mobile-Use Agent's figures, 82.1% on MobileWorld and 92.2% on MobileWorld-Real, match those on the Qwen-UI-Agent project page, which labels the MobileWorld score as GUI-only, and the MobileWorld repository noted the 82.1% result on July 30, two months before this launch. MobileWorld is a set of 201 tasks across 20 Android apps, most of them chained across several apps, and its paper shares several authors with the Qwen-UI-Agent report. When the was published in December 2025, the best system tested reached 51.7% across all its task types, which is a different measure from the GUI-only 82.1%. Qwen-UI-Agent leads every mobile chart on the project page, but the models compared there include GPT-5.6 Sol, Claude Opus 4.8 and Gemini 3.1 Pro, not the GPT-6 Astra and Claude Opus 5 used in the planner report. On the desktop benchmark OSWorld-Verified, it trails Claude Opus 4.8, 79.5% to 83.4%.

Qwen says it is opening up MobilePA-Bench, MobileWorld, MobileWorld-Real and MobileWorld-Safety. MobileWorld's repository already lets others run the benchmark and submit results, but the MobilePA-Bench site says its hidden tasks and answers remain private, with only illustrative examples public. For now, the headline numbers are Alibaba's own measurements on benchmarks built largely by Alibaba researchers. The planner repository is a report and demo site, not a release of model weights, training code or the agent itself, and the Qwen Intelligence website could not be read for this explanation, so how people get access to the agents is not covered here.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Read the source github.com