SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction
Source
Nilay Yilmaz, Naga Sai Abhiram Kusumba, Stella Wenxing Liu, Yezhou Yang
Author
Nilay Yilmaz, Naga Sai Abhiram Kusumba, Stella Wenxing Liu, Yezhou Yang
Date
Terms in this piece · Glossary
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
Multi-turn relational reasoning still sits under 75% for the strongest models, and the diagnostic separates whether a failure came from perception, missing knowledge, or context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → dropped across turns.
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li
articleThe Unwritten Benchmark: A New Challenge for Multimodal Machine Learning in Abstract Perceptual ReasoningGarima Arya Yadav, Nilay Yilmaz, Yezhou Yang
articleInter-dimension Dependence for Multi-Dimensional Evaluation of Open-Ended TextHaoyuan Li, Snigdha Chaturvedi