When Irrelevant Text Matters: Affine Margin Shifts in Multimodal Large Language Models
Source
Yinfeng Wang, Zhiyuan Yao, Zheren Fu, Lei Zhang, Zhendong Mao
Author
Yinfeng Wang, Zhiyuan Yao, Zheren Fu, Lei Zhang, Zhendong Mao
Date
Terms in this piece · Glossary
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
Irrelevant surrounding text measurably tilts multimodalA model that works with more than text — reading images, audio, or video, and sometimes generating them too.Full definition → model judgments, and it does so predictably: decision margins shift as an affine transform of their context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →-free values, so the bias can be estimated and corrected rather than merely observed.
articleCompliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System MessagesJuan Yeo, Geewook Kim
articleInter-dimension Dependence for Multi-Dimensional Evaluation of Open-Ended TextHaoyuan Li, Snigdha Chaturvedi
articleWho Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational BiasKeren Fuentes, Aaron Mueller