The Unreasonable Effectiveness of Separating the Task from the Model — Maxime Rivest & Isaac Miller

DSPy, an open-source Python framework for AI programs, separates what a task must accomplish from the model and prompts used to accomplish it. In this presentation, the speakers describe a stable contract: define the inputs and outputs, enforce essential constraints in code, and use examples to evaluate quality. That gives developers a consistent basis for comparing a new model or prompting method without redesigning the surrounding application.

An applied DSPy case study shows how evaluation can drive those changes. Its engineering account describes calibrating automated judges against human assessments, then using their feedback to propose and test changes to a chat agent's instructions. The examples and scoring criteria determine what the optimization rewards.

“in programming, if we want to repeat a task often, we make it a function. We believe the same should be true for AI programs.”

Maxime Rivest

“But if you want to do like AlphaGo or AlphaZero, and you tell them you're just going to learn from example, you're going to have a long night.”

Maxime Rivest

“Shopify, 550 times cheaper. They're able to do that because they went from an expensive model to a cheap model, but they could keep the same eval's, keep iterating on their business logic inside, and try new things.”

Isaac Miller

“Intelligence is very different from being all-knowing. If you were to ask Albert Einstein to help you with your emails, he'd probably ask what's an email.”

Isaac Miller

“You should define your problem, and you should hold your prompts, models, and code accountable to the problem that you need them to solve.”

Isaac Miller