YouTubeFeed
← AI Engineer

The Unreasonable Effectiveness of Separating the Task from the Model — Maxime Rivest & Isaac Miller

17:11 2.4K views 2026-07-23 Watch on YouTube ↗

The Unreasonable Effectiveness of Separating the Task from the Model — Maxime Rivest & Isaac Miller

Summary

Maxime Rivest and Isaac Miller, core contributors to DSPy, argue that AI programs should be built like ordinary functions: give a task a fixed input/output contract (a “signature”) and everything inside that boundary — the prompt, the model, the harness, whether it’s a single call or a full agent loop — becomes freely swappable without touching the rest of the system. DSPy is the open-source Python framework that gives this discipline its tooling. Rivest walks through his own examples — extracting tax values from farm invoices, a clipboard grammar-correction command, a full-inbox email-drafting agent built on recursive language models (RLMs) — to show the same fixed-boundary pattern scaling from tiny utilities to ambitious agentic programs.

A signature alone isn’t enough to make a task truly optimizable; Miller lays out the three pillars DSPy’s creator identified even before ChatGPT existed: specs (natural-language instructions — what should happen), code (hard constraints enforced programmatically — what must happen, illustrated with a live example that reruns extraction with more reasoning or throws on a negative tax value), and evals (examples of what good looks like — illustrated with Miller’s own story of learning to identify a maple tree from his father, not from an instruction or code, but from repeated examples). Once all three are defined, DSPy can automatically optimize the implementation — starting from few-shot example selection, moving to automatic instruction tuning, and now, in DSPy 4, extending optimization to the code harness itself via DSPy Flex. The enterprise payoff is concrete: this flexibility let Shopify cut costs 550x by swapping to a cheaper model while holding the same evals constant.

The talk closes on DSPy’s research-ecosystem model — plugging in outside innovations like Recursive Language Models (from an MIT PhD student), Jepa (a Berkeley prompt optimizer), and multi-model GRPO as one-line, signature-preserving additions — and previews “qualitative learning,” a research direction that would let models convert real-world production feedback (traces, user actions, product analytics) directly into evals they can climb, rather than requiring hand-built benchmarks. Even AGI, Miller argues, won’t inherently know how you do your emails; last-mile, context-specific learning stays essential regardless of model intelligence.

Highlights

”Shopify, 550 times cheaper”

Enterprise cost savings from swappable implementations

“Shopify, 550 times cheaper. They’re able to do that because they went from an expensive model to a cheap model, but they could keep the same evals, keep iterating on their business logic inside and try new things.” — Isaac Miller, 9:58

Clip command
yt-dlp --download-sections "*9:39-10:25" "https://www.youtube.com/watch?v=GgLQ02aO-hs" --force-keyframes-at-cuts --merge-output-format mp4 -o "shopify-550x-cheaper.mp4"

”He couldn’t give me the instruction on how to know this tree is a maple”

Why evals need examples, not just instructions

“I asked him, how do you know that this tree is a maple? And he couldn’t tell me, he couldn’t give me the instruction on how to know this tree is a maple, and he certainly couldn’t give me code on how to know this tree is a maple.” — Isaac Miller, 7:21

Clip command
yt-dlp --download-sections "*7:14-8:00" "https://www.youtube.com/watch?v=GgLQ02aO-hs" --force-keyframes-at-cuts --merge-output-format mp4 -o "maple-tree-evals.mp4"

DSPy Flex: learning the harness itself

From few-shots to prompts to learned code

“For any function that you want to implement, you can actually learn a harness over time to solve that function. And this is completely custom, and you don’t care about the implementation as long as it solves your business problem.” — Isaac Miller, 12:04

Clip command
yt-dlp --download-sections "*11:44-12:26" "https://www.youtube.com/watch?v=GgLQ02aO-hs" --force-keyframes-at-cuts --merge-output-format mp4 -o "dspy-flex-learned-harness.mp4"

”What if we could use reality to inform our evals automatically”

Qualitative learning: evals from production feedback

“Whenever you create a hill in a data set, you’re really trying to create a proxy for reality. What if instead, we could use reality to inform our evals automatically.” — Isaac Miller, 13:15

Clip command
yt-dlp --download-sections "*12:33-13:34" "https://www.youtube.com/watch?v=GgLQ02aO-hs" --force-keyframes-at-cuts --merge-output-format mp4 -o "qualitative-learning-reality.mp4"

”Ask Albert Einstein to help you with your emails, he’d probably ask what’s an email”

Even AGI needs last-mile context

“If you were to ask Albert Einstein to help you with your emails, he’d probably ask what’s an email. But, AGI will know how to do your emails. Nevertheless, it won’t know how you do your emails.” — Isaac Miller, 14:49

Clip command
yt-dlp --download-sections "*14:24-15:12" "https://www.youtube.com/watch?v=GgLQ02aO-hs" --force-keyframes-at-cuts --merge-output-format mp4 -o "agi-doesnt-know-your-emails.mp4"

Key Points

  • Functions as the model for AI programs (0:39) - Reusable, composable, testable, optimizable — treat AI tasks the way you’d treat any function.
  • DSPy brings those properties to AI workflows (1:30) - Open-source Python framework providing the tooling.
  • Why fix the contract (1:54) - New models and techniques arrive weekly; a clear input/output interface lets you swap internals with agility.
  • Concrete small examples (2:40) - Farm invoice tax extraction and a clipboard grammar-correction command.
  • Ambitious examples too (3:39) - Full-inbox drafted-reply generation via recursive language models (RLMs); “spec + repo → PR” agentic coding.
  • Fixed boundary, evolving inside (4:07) - A simple prompt becomes an agent, gains tools, gains loop engineering — all without changing the outside integration.
  • A signature alone isn’t enough to optimize (4:43) - DSPy’s creator identified three things needed to fully specify a task, even before ChatGPT existed.
  • Pillar 1 — Specs (5:22) - Natural-language instructions for what should happen; shown live in a real DSPy tax-extraction script.
  • Pillar 2 — Code (6:06) - Hard constraints for what must happen — e.g. rerun with more reasoning if extraction fails, throw on a negative value.
  • Pillar 3 — Evals (7:14) - Examples of what good looks like, for the long tail no instruction or code can capture.
  • Specs + code + evals = a fully specified goal (8:00) - Once all three exist, the program becomes optimizable.
  • Optimization has evolved (8:26) - From hand-coded few-shot example selection, to automatic instruction optimization, toward full delegation of implementation details.
  • Enterprise benefit: cheaper implementations (9:39) - Flexibility to swap models lets teams search for cheaper solutions while holding evals constant.
  • Shopify case study: 550x cheaper (9:58) - Swapped to a cheaper model, kept the same evals, kept iterating on business logic.
  • Ecosystem of pluggable techniques (10:25) - New techniques don’t guarantee a fix, but they solve sub-problems you can try for free.
  • Recursive Language Models integration (10:48) - A technique from an MIT PhD student’s paper, added as one line while the signature stays constant.
  • DSPy 4 preview: DSPy Flex (11:44) - Optimization graduates from few-shots to prompts to learning code/harnesses themselves.
  • DSPy 4 preview: Qualitative Learning (12:26) - A research direction to convert production feedback (traces, user actions, analytics) into evals automatically.
  • Even AGI needs last-mile learning (14:24) - Intelligence isn’t the same as knowing your specific context and relationships.
  • DSPy’s roots go back to 2022 (15:12) - Specs, code, and evals as a programmatic interface predate the framework’s public techniques.
  • Closing call to action (16:04) - Open source, open research, a Discord community; contribute new techniques and DSPy helps distribute them.

Mentions

Companies

  • Shopify (9:58) - Case study: 550x cost reduction by swapping models while holding evals constant.
  • cmpnd (9:16) - Isaac Miller’s company, building self-improving modular AI systems.
  • MIT (10:48) - Home institution of Alex Zheng, creator of Recursive Language Models.
  • UC Berkeley (11:22) - Source of the Jepa prompt optimizer.

Products & Technologies

  • DSPy (1:30) - Open-source Python framework for signature-driven, optimizable AI programs.
  • DSPy 4 (11:44) - Upcoming release previewed in this talk.
  • DSPy Flex (11:44) - New module type that learns a code harness, not just prompts.
  • Qualitative Learning (12:26) - Research feature converting real-world feedback into evals automatically.
  • Recursive Language Models (RLM) (3:48) - Community-originated technique for long-context AI programs, integrated as one line in DSPy.
  • Jepa (11:22) - Prompt optimizer out of Berkeley, available inside the DSPy ecosystem.
  • Better Together / multi-model GRPO (11:22) - Research technique cited as another pluggable DSPy optimization.
  • AlphaGo / AlphaZero (5:56) - Analogy for learning-from-examples versus instruction-following.

People

  • Maxime Rivest (0:12) - Speaker; DSPy core contributor.
  • Isaac Miller (0:12) - Speaker; lead maintainer of DSPy, co-founder of cmpnd.
  • Alex Zheng (10:48) - MIT PhD student who authored the Recursive Language Models paper.

Surprising Quotes

“Even if I have AGI, I would hope it doesn’t make mistakes, but whatever is in the predictor, if they make these mistakes, I still want these things to be true.” — Isaac Miller, 6:58

“None of these techniques we add will definitely solve your problem, because that’s your job. What we can do is we can solve sub-problems for you.” — Isaac Miller, 10:25

“It’s important to note, none of these techniques we add will definitely solve your problem.” — Isaac Miller, 10:25

“What’s awesome about when you build in this way… you unlock access to the collective intelligence of everyone here, all sharing techniques together.” — Isaac Miller, 16:04

Transcript

Maxime Rivest: 0:12 Please welcome to the stage, Maxime Rivest and Isaac Miller.

Maxime Rivest: 0:18 Wow, Isaac, myself, all of the DSPy community are so grateful to be here today to get to talk to you about AI programming, DSPy, and the unreasonable effectiveness of separating the task from the model, its harness, and all of the implementation details.

Isaac Miller: 0:39 When you think about it, in programming, if we want to repeat a task often, we make it a function. We believe the same should be true for AI programs. Functions are awesome. Functions are reusable, composable, testable, and optimizable. To make a function, you give it a name, you define some inputs, some outputs, and then you have some implementation logic inside of it. You get to reuse your functions thousands of times, you can optimize it, but you can also compose it into bigger programs. One of the really nice things about functions is that you can also package it and distribute it, and someone else can use it, and they just need to know about the contract on top of it to use it and they can treat it as a black box.

Maxime Rivest: 1:30 DSPy brings all of these properties to AI programs. And so DSPy is an open source software in Python that lets you, like I said, bring these properties to your AI workflows and AI programs, and it gives you all of the tooling you need to do that.

Isaac Miller: 1:54 Why do you want that? Well, we have been inventing a lot of terms in our fields in the last three years. It’s growing fast. We have new models coming every other week, we have new techniques, new strategies. And if you’re like me, you want to try all of that. But will any of these new specific techniques coming out at a different time really help on your task, on your job? Well, these are all just implementation tactics. And you want to put them inside of a clear contract. If for your repeated AI task, you define an input interface and an output interface, you get to play in the internals, you get a lot of agility.

Maxime Rivest: 2:40 Let’s make it concrete for AI. So my first AI program I made when I discovered DSPy was that I had some invoices from my farm and I wanted to extract them to do my taxes. I wanted to extract the tax values from there. Then another AI program I did is that on my keyboard and my computer I have a little command that…

Maxime Rivest: 3:00 …at read my keyboard shortcuts, read my clipboard, and will correct the grammar for me.

Maxime Rivest: 3:05 Sometimes I actually want it to also rewrite for clarity, so I have another program that takes text and just rewrites it for clarity, puts it back in my clipboard, and that’s a command.

Maxime Rivest: 3:15 And then I can, like, have a lot of agility and bring it in different places.

Maxime Rivest: 3:18 Inside of that, I can change it however I want, a new model comes out, and I can change that. It’s super easy because my interface is fixed like that.

Maxime Rivest: 3:27 I’ll skip that one.

Maxime Rivest: 3:29 But they’re not restrained to very easy things and small input-output. You can be very ambitious with AI programs.

Maxime Rivest: 3:39 So in this example, you could have your entire inbox and a new email coming in, and you want to compose a new drafted reply.

Maxime Rivest: 3:48 We can do that in DSPy with RLM, recursive language models. This is an idea that came from around our community.

Maxime Rivest: 3:57 Or, more like things we probably all do, agentic engineering or bio-coding, you can give it a spec, a repository, and you get a PR. Those are repeatable tasks.

Maxime Rivest: 4:07 And so, as I have been telling you, when you fix that boundary, you can focus on the how on the top, and then inside of it, you can have a little chat with just a simple prompt, you can iterate on that prompt, agents come out, you change it to be an agent.

Maxime Rivest: 4:21 Tools gets invented, you add tools. And then we get into loop engineering, you put that inside of it too.

Maxime Rivest: 4:31 Anything on the outside of it doesn’t change. Your integration and anything else doesn’t change.

Maxime Rivest: 4:37 And when you have such a hard boundary, you can also start to automatically optimize.

Maxime Rivest: 4:43 But how can you automatically optimize with just that simple signature? This is not enough. This is not enough to specify a task.

Maxime Rivest: 4:55 And even before ChatGPT came out, the creator of DSPy had started to land on this idea that you need three things to specify your task.

Maxime Rivest: 5:08 And if you have this language and this ability to express your task in a programming language, you can start to automatically optimize and delegate away the implementation details.

Maxime Rivest: 5:22 So the first one is what should happen. This is instructions. The signatures that I’ve been showing you are part of that.

Maxime Rivest: 5:30 Here on the screen, you see the beginning of a real script in DSPy. You set your model at the top, you configure that, and it’s fully independent of the signatures here where you have natural language instructions to extract all taxes and if it’s illegible to output zero.

Maxime Rivest: 5:46 Then you say, ‘I’m going to give you an input, it’s going to be a string, I want you to give me an output, and it’s going to be a string and a float.’ This is natural language expressing my needs.

Maxime Rivest: 5:56 This is very powerful and efficient. If you think about it, if you have a friend over coming to play a board game with you and you give them the instructions and they’re ready to play. But if you want to do like AlphaGo…

Isaac Miller: 6:00 Go or Alpha Zero, or Alpha Zero, and you tell them you’re just going to learn from example, you’re going to have a long night.

Isaac Miller: 6:06 And then the second one is what must happen. There are some constraints you have that they have to be listened to, they have to be enforced. The best way to do that is with code.

Isaac Miller: 6:20 So, I want you to go to the third line, fourth line. You have self dot extract and self dot recheck. You can see where we’re doing a predict on the extract axis and we’re doing a chain of thought on the extract axis.

Isaac Miller: 6:30 The first one is a vanilla program, the second one makes it do some reasoning. Now I’m taking them inside in the forward,

Isaac Miller: 6:41 and you can see in the if not present tax, this is a requirement I have that if my first simple vanilla program doesn’t extract my taxes, I want you to rerun with more reasoning. I mean, I got to get my taxes right.

Isaac Miller: 6:58 And then another requirement I have is if the value is below zero, throw. I want to show that to a human. I don’t want to let you go. This will not change. Like, even if I have AGI, I would hope it doesn’t make mistakes, but whatever is in the predictor, if they make these mistakes, I still want these things to be true.

Isaac Miller: 7:14 So the last one is what good looks like. And when I was young, I was on the farm with my dad

Isaac Miller: 7:21 and I asked him, how do you know that this tree is a maple? And he couldn’t tell me, he couldn’t give me the instruction on how to know this tree is a maple, and he certainly couldn’t give me code on how to know this tree is a maple.

Isaac Miller: 7:34 And so through time, with examples, I learned how to know that a tree is a maple. But this is not limited to things like classifying plants.

Isaac Miller: 7:46 It’s also for all of the long tails in your specifications that are things that are more latent. These are sometimes a reason why you would do internship and you would have a mentor and a mentee. You’re looking at a lot of examples and there are long tails of successful behaviors that you have to see and learn.

Isaac Miller: 8:00 Now that you have all of these, you have expressed fully, you have all these three languages you can put together, you have the specs, the code, and the evals, and now your goal is fully specified.

Maxime Rivest: 8:12 And so you can start optimizing. You can use things like DSPy on your metrics and on your program, and you can start optimizing.

Maxime Rivest: 8:26 At the beginning of the DSPy, the ChatGPT didn’t exist. The models were not good enough to optimize, and so we were using code to find few-shot examples to make the base models act in the proper way.

Maxime Rivest: 8:40 Then models got better, and so we could automatically optimize instruction.

Maxime Rivest: 8:50 And in the future, we are starting to be able to be liberated more and more from the implementation details and delegate that away. And at the end, our hope in the DSPy is that you can stick to all of that and then just the news…

Maxime Rivest: 9:00 implementation details will be automated for you. Isaac will talk to you a lot more about what has been released in the last year, what we’re releasing now, and all of the future plans we have. Thank you.

Isaac Miller: 9:16 Thanks Max. So, we’ve given you a pretty big abstract overview of specs, code, and evals. But these aren’t things that are just restricted to the academic sphere. These are used in production by some of the biggest enterprises for massive gains. And we see two main benefits when you use DSPy in the enterprise.

Isaac Miller: 9:39 First is that your implementation becomes cheaper. When you’re flexible to what the implementation is, you can use the bitter lesson to search over different solutions, find something that solves your problem cheaply. And you can use this to scale to data sizes that weren’t possible with a more expensive implementation.

Isaac Miller: 9:58 Shopify, 550 times cheaper. They’re able to do that because they went from an expensive model to a cheap model, but they could keep the same evals, keep iterating on their business logic inside and try new things. There’s three awesome case studies here and you should check them out after the talk. They give a lot of details on how you can do this in your own enterprise.

Isaac Miller: 10:25 Now, part of the reason why you want to build in the DSPy ecosystem is that we’re constantly adding new techniques for you to try. And it’s important to note, none of these techniques we add will definitely solve your problem, because that’s your job. What we can do is we can solve sub-problems for you that make your implementation easier.

Isaac Miller: 10:48 For instance, Alex Zheng, a PhD student at MIT, came out with this paper called Recursive Language Models. Recursive language models are a way to solve some kinds of long context programs. And guess what? We can bring this into DSPy for you to try. See if it helps your long context tasks. Maybe it will, maybe it won’t. But the thing is, it’s one line and your signature stays the same. That’s what’s important here. Everything gets to stay constant and you get to see if this solves your problem or not.

Isaac Miller: 11:22 And we’ve had a number of examples of this just in the last year from people building in and around the DSPy community. We’ve had RLM’s, we’ve had Jepa, which is an incredible prompt optimizer out of Berkeley. Better Together, multi-model GRPO. All these are incredible research innovations that you get to try in your implementation just by being in the DSPy ecosystem.

Isaac Miller: 11:44 And we have more coming in DSPy 4. And I’m excited to talk to you about two of those today: DSPy Flex and Qualitative Learning. DSPy.flex is a new kind of module. In DSPy, when we let you optimize things, it started with few-shot examples. Then it became

Isaac Miller: 12:00 Prompts. And now, that’s becoming code.

Isaac Miller: 12:04 For any function that you want to implement, you can actually learn a harness over time to solve that function.

Isaac Miller: 12:11 And this is completely custom, and you don’t care about the implementation as long as it solves your business problem, which you’ve created ways to measure because you’ve defined the three core parts of specs, code, and evals.

Isaac Miller: 12:26 The second thing I’m excited to talk about is qualitative learning.

Isaac Miller: 12:33 One of the hard-hard problems in AI engineering is building evals. And there’s a few reasons why this is hard.

Isaac Miller: 12:41 One is that defining what good looks like is really challenging for any real-world problem.

Isaac Miller: 12:48 The second is that when you define good, oftentimes you have to lose detail. If an email is good or bad contains a lot less information than if you know what could change in that email in order to improve.

Isaac Miller: 12:59 And the third is that whenever you create a hill in a data set, you’re really trying to create a proxy for reality. What if instead, we could use reality to inform our evals automatically.

Isaac Miller: 13:15 What qualitative learning asks is how do we decrease this question? How do we decrease this distance? And it’s a research question right now. But what we believe is that models are now good enough to interpret whatever textual feedback is present in the environment and convert that into evals and a hill that the model can climb.

Isaac Miller: 13:34 And so as you get more feedback from production, its traces, its user actions, its product analytics, its the model asking you questions about how data should be represented. As you do this, the model can iteratively refine the hill over time and continue climbing it to solve your actual business problem.

Isaac Miller: 13:57 And DSPy focuses on these kinds of last-mile problems. We’ve got a really strong research ecosystem, and we collaborate really closely with them. And that’s part of the beauty is that we can see the problems that happen in applied AI engineering, define them, build a benchmark, and then solve them with techniques. And then we get to democratize the results of that to everyone because it’s open-source, open research.

Isaac Miller: 14:24 Now, one common question is what happens when we have AGI? Well, even when we have an incredibly smart model, the model won’t know how to solve your problems, it won’t know how to do your tasks or have your context.

Isaac Miller: 14:39 And so this genre of last-mile learning is trying to ask how do we efficiently do this learning? Intelligence is very different from being all-knowing.

Isaac Miller: 14:49 If you were to ask Albert Einstein to help you with your emails, he’d probably ask what’s an email. But, AGI will know how to do your emails. Nevertheless, it won’t know how you do your emails.

Isaac Miller: 15:00 As it won’t know how to actually solve your problem and interact with the people you need to interact with. It won’t understand your relationships without learning this context over time.

Isaac Miller: 15:12 Since 2022, DSPy has been focused on these three core ideas of specs, code, and evals, all defined as a programmatic interface.

Isaac Miller: 15:24 We’ve certainly evolved over time and new techniques are incredible. We’ve gone from evolving few-shots to prompts to harnesses and now evolving your evals over time too.

Isaac Miller: 15:37 But what you need to ask for any of these new techniques is how do they help you solve harder problems or solve your own problems better?

Isaac Miller: 15:47 And you should ask this question in a data-driven manner. You should look at this new technique, say, how can I apply this to the business problem that I have? You should define your problem and you should hold your prompts, models, and code accountable to the problem that you need them to solve.

Isaac Miller: 16:04 And what’s awesome about when you build in this way where you have flexible implementations, what you unlock is you unlock the ecosystem of all the techniques that anyone in this room is constantly inventing. You unlock access to the collective intelligence of everyone here, all sharing techniques together.

Isaac Miller: 16:29 So, if you want to build reliable AI software, I encourage you to come check out DSPy. We’re completely open source, open research, and we’re here to help you solve your problems by building reliable software. We have a Discord that you should come join. And when you come up with the next technique, you should come contribute it to DSPy, and we can help you distribute it and make this awesome technique available for everyone. Thank you.