Intel / article

ZeroR@CHiPSAL 2026: Two-Stage Vision-Language Adaptation with Contrastive Learning for Nepali Meme Classification

Source: arxiv.org
Author: Nitiz Khanal
Date: Published Aug 3, 2026

Context

A system for classifying Nepali memes processes their images and Devanagari text together, avoiding a separate text-extraction and translation pipeline. The ZeroR study adapts Qwen's Qwen3-VL-8B-Instruct model in two training stages and combines their classification scores for the final decision. The authors report second place for hate-speech detection and fourth place for sentiment analysis in the CHiPSAL 2026 shared task. The contribution is a tested approach to analyzing mixed visual and written content in Nepali, with results specific to that competition. Direct processing removes an intermediate translation step, but does not establish that classification errors or bias disappear.

Key takeaways · AI-distilled

The two training stages do different jobs. with an MLP head teaches the model to emit the label; a second contrastive pass with supervised InfoNCE reshapes the backbone's so the classes separate.
Class imbalance gets attacked from three sides at once: oversampling the minority class, augmenting the images, and focal loss so easy examples stop dominating the gradient.
At they blend stage-one token probabilities with stage-two classifier scores using weights tuned on the validation set, rather than trusting either head on its own.
Binary hate detection reached 0.797 F1 while three-class sentiment reached only 0.518. The same adaptation recipe degrades sharply as the label space gets finer and more subjective.

Terms in this piece · Glossary

LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

Key quotes

“Our system achieved \textbf{2nd place} on hate speech detection (F1: 0.797) and \textbf{4th place} on sentiment analysis (F1: 0.518).”

“Our end-to-end approach eliminates error propagation from separate OCR and translation pipelines by leveraging the model's native Devanagari understanding.”

Read the source arxiv.org