Intel / article

How we used DSPy to turn AI evaluations into better responses in Dash chat

Source: dropbox.tech
Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon
Date: Published Jun 25, 2026

Context

Dropbox improved Dash chat's prompts in two stages: first calibrating the AI evaluators against human judgments, then using those evaluators to guide changes to the chat agent. The evaluators inspect the agent's searches, and supporting evidence as well as its final answer. Dropbox's account describes using DSPy, a prompt-optimization framework, to connect those evaluations to proposed revisions.

Candidate prompts are replayed on representative historical internal chats, making recurring failures available as feedback for the next revision. Dropbox reports a 26% reduction in incomplete answers within the first 24 hours of launch. That is a company-reported short-window result; the reusable mechanism is testing prompt changes against realistic interactions with evaluators already checked against people.

Terms in this piece · Glossary

tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.

Read the source dropbox.tech