Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

Chinese CLIP is designed to match Chinese descriptions with relevant images. The Qwen team argues that translating a query into another language can lose meaning, while image collections themselves can differ across cultural settings. Its method adapts both sides of the matching system.

Training first keeps an existing image encoder fixed while teaching a Chinese text encoder to produce compatible representations. It then trains both together on image-text pairs so the visual component can also adapt. The team reports stronger retrieval results on three test datasets, with its largest advantage on the Chinese-native MUGE dataset. This supports adapting the image data as well as the language component for Chinese image search.

“It plays not only as a foundation model but also a bridge between vision and language.”

“We therefore launched this project to promote the Chinese multimodal representation learning.”