MiMo-VL-7B posts GUI grounding numbers that beat models built for the job
- Source
- x.com
- Date
Today, MiMo can see We release MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing even Qwen2.5-VL-72B and GPT-4o . For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. To facilitate the evaluation of multimodal tasks, we release a comprehensive evaluation suite covering over 50 tasks to promote reproducibility and advance the field, which will be open-sourced soon. Our paper, model and code can be accessed via the following link. Github: https://t.co/aatkHFZ7Oz Huggingface：
What can MiMO-VL-RL do? 🤔 In Case #1, our model showcases strong plot understanding capabilities, successfully converting an intricate plot into a well-structured markdown table. It also demonstrates superior reasoning capabilities in STEM tasks. In Case #2, MiMo-VL-7B successfully navigates a webpage to complete the task of purchasing a Xiaomi SU7 with customized paint and interior options.
The comprehensive visual perception capabilities of MiMo-VL-7B are attributed to high-quality pre-training data and an innovative Mixed On-policy Reinforcement Learning (MORL) algorithm: Multi-Stage Pre-training: High-quality pre-training multimodal data has been collected, cleaned, and synthesized, covering data types such as image-text pairs, video-text pairs, and GUI operation sequences, totaling 2.4T tokens. By adjusting the proportions of different types of data in stages, it enhances the ability for long-range multimodal reasoning. Mixed Online Reinforcement Learning: It combines various feedback signals including mixed text reasoning, multimodal perception + reasoning, and RLHF, and employs online reinforcement learning algorithms to stabilize and accelerate training, significantly improving the model's reasoning, perception performance, and user experience.
On general benchmarks, the MiMo-VL-7B models, particularly MiMo-VL-7B-SFT and MiMo-VL-7B-RL, demonstrate consistently leading performance across a diverse range of vision-language benchmarks, surpassing other open-source models of comparable or larger scale.
Context
Xiaomi's MiMo lab released MiMo-VL-7B-SFT and MiMo-VL-7B-RL, reporting that the RL version outperforms Qwen2.5-VL-7B on 35 of 40 evaluated tasks and scores 59.4 on OlympiadBench, ahead of the much larger Qwen2.5-VL-72B and GPT-4o on that specific .
On GUI , locating and acting on on-screen elements, the company reports 56.1 on OSWorld-G, ahead of UI-TARS, a model built specifically for that task. Xiaomi attributes this to 2.4 trillion of multi-stage that includes GUI operation sequences, combined with a reinforcement learning method it calls Mixed Online Reinforcement Learning. These are Xiaomi's own reported scores, not an independently run comparison.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
Checking sign-in…
Loading comments…