From Scratch to SOTA: Training a 3B State-Space Vision Model — Krishna Prasad Srinivasan, Sarvam

• Sarvam's 3B document model uses a state space backbone instead of a transformer because a single page can run to about ten thousand visual tokens, where quadratic attention gets expensive and an SSM keeps constant memory.
• The team chose block-level OCR wrapped in separate layout and reading-order harnesses rather than a monolithic page-level VLM, a design Krishna Prasad Srinivasan says many 2026 releases later converged on.
• Training ran in four stages: 13 trillion text tokens (English, 22 Indian languages, math, code) before any images, then 300M image-text pairs, 100M OCR fine-tuning samples, and finally reinforcement learning.
• Because OCR output is machine-checkable, Sarvam's RL rewards are unit tests on character error rate, table structure, or grammar, which the speaker says makes verifiable-reward RL scale for this task.
• The talk frames Indic document AI as a data gap: well under 1% of Common Crawl is in an Indian language, and much of the knowledge was never digitized. The model is reportedly processing 35M pages for insurers, banks, and governments.

• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
• attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
• fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.

Shows a concrete architectural bet (SSM backbone for constant memory on long visual-token sequences) and a training recipe that let a small model outperform far larger competitors on a genuinely underserved OCR task.

• videoFrom Ingestion to Agents: How AI Teams Build on Document Intelligence — Adit Abraham, Reducto
• videoYou’re Not Thinking Big Enough: Rebuilding Food Systems with AI Agents — Cody Menefee, Firecrawl
• videoFrom VLM/VLA's to Embodied Agents — Armen Aghajanyan, Perceptron AI
• videoSkill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face

• videoSkill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging FaceAI Engineer
• videoLarge clusters for small models — Daniel Svonava, SuperlinkedAI Engineer
• videoTraining Krea 2: What matters in generative model training — Sangwu Lee, Krea.aiAI Engineer

Checking sign-in…

Loading comments…