Scale AI
Scale AI provides data infrastructure, evaluation and AI applications for organizations building and using AI.
X / scale_aiNotable people
People map
Current peopleNotable public-facing people we cover. Grouped by public roles, not reporting lines; this is not a complete staff directory.
Leadership
- Alexandr Wang
Co-founder and board member · 2016 – Present
Former people
Writing, research & talks
Scale details post-training a Nemotron model for a French-language insurance call copilot, split into fact extraction, coverage recommendation and over-promise detection. Reported results: 93.8% extraction accuracy, 95.6% recommendation accuracy, and 70 to 75% lower inference cost.
Scale reports that on a multi-agent enterprise deployment, automated attacks succeeded 3% of the time while human red teamers succeeded in 68% of sessions, and ordinary users triggered violations 61% of the time. It lays out five system layers to test and a test, patch, re-test cycle.
- Hitl Routingscale.com
Scale describes an agentic accounts-payable credit-recovery system and a framework for setting human-in-the-loop thresholds by pricing mistakes against reviewer time. It also notes that regulated decisions are handled as fixed rules rather than priced.
- Insights Generatorscale.com
Scale introduces Insights Generator, which reads across an agent's full run history to name failure patterns. In its study, engineers using it improved agents by 30.4 points versus 16.2 with Claude Code subagent reports. It also documents agents misreporting success on benchmarks.
- Veroscale.com
Scale's VeRO framework lets optimizer agents edit any part of a target agent. Across 105 runs gains reached 19 points on GAIA, concentrated in tool use and workflow logic, and structural fixes held up when the underlying model changed while prompt edits did not.
- Drugdiscoverybenchscale.com
Scale Labs and Phylo's DrugDiscoveryBench ranks top agents on 82 expert-written computational drug discovery tasks. The best solve about half; failures cluster in 7-8 step chains where agents lose the original question, while expert plans and extra thinking time help.
- Data Qualityscale.com
Scale's applied research team outlines a three-level data quality system. The task-level piece, Archie, uses AI agent judges with human review of edge cases, auto-generated eval tasks, and iterative prompt tuning against calibration data to keep judgments aligned with humans.
- Aspiscale.com
Scale Labs' ASPI benchmark finds most frontier models are far more prone to prompt injection once an agent pauses to ask a clarifying question, up to 10x for some. Prompt guards and tool filters reduce attack success but do not close the gap.
- Swe Atlas Completescale.com
Scale's completed SWE Atlas covers Codebase QnA, Test Writing and Refactoring over 284 tasks. Top agents cluster in the 40s, none pass 50%, and reliability across three attempts lags single-attempt success by two to three times.
- Hilscale.com
HiL-Bench adds hidden information gaps to SWE-Bench Pro and BIRD tasks and measures whether agents ask for help. Pass rates collapse from 75-89% to 4-38%, and GPT, Claude and Gemini each fail differently. Code and dataset are public.
- Llm Novice Upliftscale.com
A Scale Labs and SecureBio study ran novices with frontier LLMs against internet-only controls on 10 in silico biosecurity tasks. LLM users outperformed controls, exceeded experts on several benchmarks, and often got past model refusals. Summarised from the blog post, not the paper.
- Swe Atlasscale.com
Scale launched SWE Atlas, a three-part benchmark suite for coding agents in real repositories. Codebase QnA is live now, scoring runtime-grounded investigation across architecture, root-cause, security and API tasks. Test Writing and Refactoring follow.
- Rok Fortress Multilingual Ai Safetyscale.com
Scale and Korea's AI Safety Institute built ROK-FORTRESS, which varies prompt language (English or Korean) and geopolitical grounding independently across 1,235 tasks and 14 models, with benign counterparts to measure over-refusal.