ERNIE 5.0: A 2.4 Trillion-Parameter Unified Multimodal Foundation Model

ERNIE 5.0 puts text, images, video and audio into a shared model for understanding and generation. Baidu describes training these forms of information together from the outset, with images treated as single-frame videos and audio learned from representations spanning meaning and acoustic detail. The intended benefit is that relationships across media become part of the model's training, rather than depending entirely on separately added output components. The architecture announcement explains this design.

Its 2.4 trillion parameters describe total capacity. A routing mechanism selects a small fraction for each input unit, drawing from a shared pool of computational experts across media. That distinction explains how the design seeks broad capacity without using the entire model for every calculation.