From Scratch to SOTA: Training a 3B State-Space Vision Model — Krishna Prasad Srinivasan, Sarvam
Source
youtube.com
Author
AI Engineer
Date
Context
Sarvam Vision combines a document-reading model with separate components for page layout and reading order. The aim is to recover usable text from scanned records in English and 22 Indian languages while preserving how the page is organized.
The training talk describes recognizing text in individual blocks rather than asking one model to reconstruct an entire page at once. It also describes teaching the model language before introducing images, so that linguistic knowledge can help interpret unclear writing. This makes the contribution a coordinated document-processing system: recognizing words and reconstructing their relationship on the page are distinct jobs.
Key takeaways · AI-distilled
Sarvam's 3B document model uses a state space backbone instead of a transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition → because a single page can run to about ten thousand visual tokens, where quadratic attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → gets expensive and an SSM keeps constant memory.
The team chose block-level OCR wrapped in separate layout and reading-order harnesses rather than a monolithic page-level VLM, a design Krishna Prasad Srinivasan says many 2026 releases later converged on.
Training ran in four stages: 13 trillion text tokens (English, 22 Indian languages, math, code) before any images, then 300M image-text pairs, 100M OCR fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → samples, and finally reinforcement learning.
Because OCR output is machine-checkable, Sarvam's RL rewards are unit tests on character error rate, table structure, or grammar, which the speaker says makes verifiable-reward RL scale for this task.
The talk frames Indic document AI as a data gap: well under 1% of Common Crawl is in an Indian language, and much of the knowledge was never digitized. The model is reportedly processing 35M pages for insurers, banks, and governments.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
videoYour Financial Agent Needs to Show Its Work - Anirudh Veeraragavan, Fieldguide
videoBuild Your Own Finance Agent Inside ChatGPT - Stefano Fabbri & Oliver Steffmann, OpenAI
videoDon't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal
videoHow to Get Frontier Coding Results at 70% Lower Cost - Nouha Dziri, Cohere
