Jess Wang

Wang previously worked as a software engineer at Microsoft and DoorDash, then in developer relations at Warp. At the start of 2026 she announced her move from Warp to Braintrust, explaining that a growing interest in evaluation had exposed a gap in educational material. She now presents evaluation methods through talks, writing and a podcast co-hosted with Chris Pennington.[2][3]

Make a search comparison inspectable

Wang evaluates bug-fix pull requests in Microsoft’s TypeScript-Go repository. It checks out the parent revision, derives a task from the fix and compares filesystem exploration with vector search. It records test outcomes and cost, then examines repeated searches and missing calling context. She describes the small dataset, weak vector implementation and need for repeated trials as limits on the result.[1]

Distinguish a clean trace from a completed task

In her Hugging Face article, Wang imports agent traces into Braintrust, normalizes their structure and adds task-success judgments. An example agent removes existing contacts but never adds the required new contacts, despite finishing without tool errors. The example explains why a success label needs to examine what happened, not just whether an execution ended cleanly. Her article also distinguishes stronger evidence from tasks whose hidden state the judge cannot inspect.[4]

Wang frames evaluation as a dataset, a task, a scoring method and a comparison. Production traces supply new cases when users reveal a failure. Dev Convos extends her teaching into discussions of developer tools and AI research.[1][2]