Intel / article

Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding

Source: arxiv.org
Author: Reza Khanmohammadi, Simerjot Kaur, Charese H. Smiley, Ivan Brugere, Mohammad M. Ghassemi
Date: Published Aug 10, 2026

Context

A confidence score can rank financial-chart answers reasonably well without accurately describing their chance of being wrong. Researchers compared seven confidence estimators across five vision-language models and found that several -only methods were overconfident despite useful rankings. Learned estimators using internal model signals produced scores better suited to deciding which answers required human review. Research abstract

The amount of work that could pass such a review threshold still depended on the underlying model's competence. On the hardest tested condition, almost no answers qualified for automatic acceptance under a strict 5% error budget. The learned probes had been trained on natural images and transferred to finance without adaptation, making this a test of transfer across domains. Evaluation scope and findings

Terms in this piece · Glossary

inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.

Read the source arxiv.org