Intel / article

DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs

Source: arxiv.org
Author: Yingxuan Zhuang, Miao Pan, Wangjie Gan, Jingxiao Yang, Fan Wang, Weiming Liu, Cheng Tan, Xuhong Zhang, Jintao Chen
Date: Published Sep 25, 2026

Context

DEEPO targets two reasons reinforcement learning can struggle to correct false claims about images and videos. When every sampled answer is wrong, comparing their rewards supplies little direction for improvement. Separately, very confident predictions can receive weak corrective updates.

The method adds partial reasoning excerpts, with the final answer withheld, to training on difficult questions, and adjusts update strength to counteract the weak correction of confident errors. The excerpts provide supervision and can help produce a successful answer against which failures are compared. The authors report improved -related benchmark performance; the excerpts are used during training only and are not supplied when the trained model answers new questions.

Key takeaways · AI-distilled

The authors trace uneven RL results to two weak points. First, hard queries with high semantic entropy often yield sample groups that are all wrong, collapsing the group-relative advantage to zero exactly where hallucination risk is highest.
Second, confident-but-wrong barely get corrected: a categorical policy's expected score-gradient norm vanishes as its distribution sharpens, so the predictions most in need of correction receive the weakest updates.
DEEPO adds expert prefixes with continuations on high-uncertainty queries to restore advantage variance, and applies advantage-sign-aware Renyi preconditioning so correction reaches confident errors.
Each branch improves on GRPO alone. Their interaction is statistically significant on VideoMMMU (+4.0, 95% CI 1.1 to 6.9) and additive on the other tasks.

Terms in this piece · Glossary

hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.

Read the source arxiv.org