What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team

Handshake AI sells frontier labs expert-made data and the reinforcement-learning environments that train and test AI agents. That is how Latent Space's episode description frames the business, which it says went from launch to $1 billion in gross annualized revenue in roughly a year. An environment here is a simulated workplace the agent has to act inside, rather than a list of questions with answers. The show notes do not say what any particular lab bought, so the clearest view of the product comes from two benchmarks Handshake has published. Both are built from material in professionals' actual work rather than polished textbook problems, and in both, Handshake's write-ups trace why a model fails, which a single score does not show.

ATLAS-Finance puts an agent inside one of 13 simulated financial firms, each with inboxes, chat, shared files, notes and calendars, and dozens of coworkers and clients. Its 100 tasks were built by bankers up to vice-president level from institutional banks and boutiques, and they take a human expert 15 to 30 hours. Nobody hands the agent a clean question. It has to piece together its assignment from email, settle contradictions by finding the governing source or the person with the final say, sometimes absorb a correction that arrives mid-task, and then build a financial model and send it to exact recipients under an exact filename. Handshake argues that existing finance benchmarks miss this because they pose static, fully specified tasks.

The grading checks how the workbook was built. Each task carries 42 to 486 yes-or-no criteria written by experts, and the criteria are chained: if an early schedule is wrong, everything that depends on it is marked unreachable instead of earning credit for correct arithmetic on a bad number. Handshake's walk-through of one deal, Project Osprey, shows why. A Claude Opus 5 agent worked for 73 steps, produced a workbook with 2,641 live formulas and zero error cells, saw all thirteen of its own integrity checks pass, and still failed. A client email listed existing goodwill without saying what to do with it, and the Python version of the model the agent wrote to check its work silently subtracted it from the tax basis, cutting each year's tax amortization by roughly a quarter. Because every self-check compared the workbook against that same Python model, the checks confirmed the error. According to the show notes, the episode shows a similar mistake on a different deal, Project Falcon, where Collin Schweiker points out that the model can explain the tax treatment when asked directly; it fails to recognize the line of code where the rule applies. Across 11 frontier models, the best passed 12.3% of tasks in version 1.0.0 of the benchmark, run in September 2026, and the benchmark's results page, which reports the latest version, puts the top pass rate at 10.67% while the leading models still score about 70% on partial credit. In Handshake's own words, agents complete substantial portions of the work but "rarely produce correct deliverables." Its analysis of 1,100 runs traced 53.8% of failures to applying the wrong financial treatment, convention or timing, and only 0.1% to picking the wrong source document or input.

Handshake's biology benchmark points to a different weak spot. VIALS asks 161 questions about the images scientists handle in lab work, such as gels, microscopy, plasmid maps and flow cytometry plots, and scores each against a fixed correct answer rather than a rubric. Handshake reports that most PhD-level biomedical scientists find the tasks straightforward, while the best model on its chart, GPT-6 Astra, scores 49.9%. In the brain-tissue example described in the show notes, Nick Magazine picks two of nine slides as showing the most advanced vessel necrosis; the model picks those two and adds a third. Here much of the trouble is in seeing. When Handshake post-trained NVIDIA's open Nemotron 3.5 Super VL model, it found that replacing each image with a high-quality written description let the model answer as well as GPT-6 Astra, so the gap lay in reading the image rather than reasoning about it. Fine-tuning on 2,000 Handshake-made examples with image descriptions doubled its VIALS accuracy, from 7.07% to 14.13%, with no drop on Humanity's Last Exam or MMMU-Pro.

A single pass rate or accuracy figure hides both kinds of failure. What this material gives a lab is a diagnosis: ATLAS-Finance reports which schedule broke and what that cost downstream, and the episode's coding segment, by the show notes' account, inspects which files an agent read, whether it checked the copies and what it did to the shell history, most of which a transcript of the agent's final reply would miss. The Nemotron case shows why that matters: once Handshake saw the model failed at seeing, it moved from reinforcement learning alone to supervised fine-tuning on examples paired with image descriptions. The limit is that every benchmark result here comes from Handshake, which also sells the data meant to close these gaps, and the episode accounts come from its published show notes. Mercor, whose APEX-Agents benchmark appears in ATLAS-Finance's comparison table of finance and professional-work benchmarks, also builds agent benchmarks, and Mercor's Brendan Foody explained how RL environments teach agents real-world work.