VisRAG

Why it's ranked here

The project tackles a real weakness in document question answering: text extraction can discard layout and other visual information. Its strongest case is the explicit evidence stage for multi-image reasoning, backed by released models, training procedures, datasets, evaluation scripts, and several generation modes. The setup remains research-oriented and operationally demanding.

What's good

The evidence format makes the reasoning pipeline inspectable. Each image receives a relevant finding or an explicit no-information marker before answer generation. Users can evaluate exact match, accuracy, and F1, including separate treatment of answerable and insufficient-evidence cases. Retrieval training supports distributed contrastive learning, shared negatives, gradient caching, and custom datasets.

Tradeoffs

Installation pins an older CUDA toolkit and several exact package versions, then requires editable installation of both the main package and a modified vision library. Training spans external frameworks and manually prepared datasets. The supplied prediction script contains placeholder model locations and fixes visible GPUs in code. Generation still depends on substantial vision-language models and local orchestration.

How to use it well

Choose VisRAG for research or controlled document question-answering systems where charts, slides, page layout, and evidence across several images matter. Start with the released retriever and vision-language models, then benchmark against your own documents and insufficient-evidence cases. It does not provide a hosted application service, general document management, or a lightweight turnkey deployment surface.

Observed

Primary language

Python

Main package

Setuptools package named openmatch-thunlp, installed from the repository in editable mode

Install surface

Conda environment, pip requirements, two editable package installs, and an NVIDIA CUDA 11.8 toolkit

Interfaces

Python library modules and command-line scripts for training, prediction, generation, retrieval, and evaluation

Model ecosystem

Released retriever and generation models are linked through Hugging Face

Licensing metadata

The main package classifier declares MIT; the bundled RS-GRPO package declares Apache 2.0