Intel / article

SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction

Source: arxiv.org
Author: Nilay Yilmaz, Naga Sai Abhiram Kusumba, Stella Wenxing Liu, Yezhou Yang
Date: Published Aug 31, 2026

Context

SciReC investigates why a model fails to connect information across academic figures in a continuing conversation. After a weak relational answer, its adaptive follow-up questions probe whether the model remembers the details and whether showing the image again helps. The paper uses those responses to separate memory, visual understanding and knowledge problems from difficulty combining the facts.

Its diagnostic framework attributes the largest share of failures to relational reasoning, followed by memory. Those shares come from fixed scoring formulas applied to judge-assigned scores, and relational reasoning is the residual left after the other components are accounted for, so they are not direct observations of what happens inside the model. The offers a structured way to investigate failures that a single answer score leaves unexplained.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.

Read the source arxiv.org