Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Source
youtube.com
Author
AI Engineer
Date
Why it matters
A reusable pattern for high-stakes outputs: separate retrieval, deterministic computation and verification so the model never authors a figure. Includes evidence that plain RAG produced thousands of wrong numbers.
Key takeaways · AI-distilled
In Kepler's architecture, as CEO Vinoo Ganesh describes it, models cite financial figures but never write them: retrieval finds the source, deterministic code does any math, and a post-processing filter strips figures that can't be verified.
Kepler's citation registry is built from more than 30 million SEC documents, and each extracted fact is structured by value, unit, scale and period so every number stays traceable to its source.
Kepler retrieves with hybrid searchCombining keyword and semantic search in one ranked result, so exact terms and paraphrases both find what they should.Full definition → and re-ranking on turbopuffer and isolates each tenant's data; Ganesh contrasts this with plain RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition →, which produced thousands of wrong numbers in Kepler's evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →.
Terms in this piece · Glossary
hybrid search — Combining keyword and semantic search in one ranked result, so exact terms and paraphrases both find what they should.
reranking — A second pass that re-scores retrieved candidates by reading each one against the query, fixing the ordering that fast vector search got approximately right.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
videoWhy Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha
videoAI Writes the Code. So What's Left for Engineers? - Laurie Voss, Arize AI
videoHow to Build Financial Agents That Work Across the Web - Jay Sahnan & Arik Bird, Browserbase
videoYour Agent Fix Just Broke Something Else - Sofia Jakovcevic, Arize AI