Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp

The smaller model that beat Opus is a specialist with one job. Inside Ramp Sheets, Ramp's spreadsheet agent, the main model hands questions such as "What was the revenue from March to May?" to a helper called Fast Ask, which opens the workbook, reads the relevant ranges and returns a short answer. A model that does a narrow task on behalf of another model like this is called a subagent. Ramp Labs, the company's applied research group, built it because its traces showed the main agent spending 17.8% of its tool calls opening tabs and reading ranges, and about three quarters of those calls were followed straight away by another read, a sign it often missed the right data on the first try (Ramp Labs write-up).

Fast Ask is a retrained Qwen model, Qwen3.5-35B-A3B, which is open-weight: its parameters are published, so Ramp could run and modify it itself. Only about 3 billion of its parameters are active for each step of output, which keeps it fast. Working with Prime Intellect, Ramp post-trained it with reinforcement learning. Instead of showing the model correct examples, Ramp let it attempt each question eight times against synthetic finance workbooks, the harder ones seeded with decoy tabs and misleading shortcuts, then scored every attempt. An exact correct answer earned almost all of the reward, with small bonuses for using fewer turns and replying concisely. That works because spreadsheet lookups usually have one right value, such as a date, an invoice ID or an amount in cents, so a program can grade each attempt without a human.

"Beat Opus" means this: on Ramp's held-out set of those synthetic workbook questions, Fast Ask scored about 4 percentage points higher than Claude Opus 4.6 from Anthropic on exact-match accuracy, and it ran at about the speed of Anthropic's smaller Haiku 4.5. Reinforcement learning added 10 points over the same Qwen model before this training. Ramp ran this evaluation itself, on tasks it generated, with every model in non-thinking mode to match the low-latency setting it uses in production. It shows a trained small model outperforming Opus on spreadsheet retrieval, not on financial reasoning in general, and the video description's phrase "financial question answering" is broader than what the write-up measured. Ramp's own conclusion is the narrower one: train small subagents for narrow, frequent, checkable bottlenecks and let frontier models spend their tokens on judgment.

The talk's third project applies the same cost reasoning to security. Ramp pointed roughly 10,000 coding agents at its backend and, after other agents removed duplicates and reproduced the results, confirmed seven high-severity vulnerabilities, all since patched. That scan ran on GPT-5.5, and a run that size can exceed $20,000 at full price. When Ramp retested the seven bugs with Kimi K2.6 and DeepSeek V4 Pro, open-weight models it describes as roughly five times cheaper per token, GPT-5.5 was the strongest overall and the only model to catch a couple of the subtler bugs. Kimi stayed close to it on most of the seven and DeepSeek was weaker across the board, though both still found some at meaningful rates (security write-up). In both projects, Ramp's research gives it its own measurements of where a cheaper model is good enough and where a frontier model is still worth paying for. The talk's second project, KV cache "cartridges" that preload context for tasks such as month-end close, is described only in the video description; none of the Ramp write-ups reviewed here document it.