Clip transcript
in these varied or diverse orgs like how to solve. Um, so making a code base agent ready, right? This is a very challenging thing to do. Uh, most organizations have some degree of consistency in how they've chosen to build deterministic validation loops inside of their company, right? So, your code base runs linters, type checkers, uh, it might run some security scans, and it's like check mark. Like, it passes or it doesn't. The end end test, they pass or it doesn't, right? Or they don't. Um, what agent readiness really is is it's a measure of how many of these deterministic validation loops are present inside of your code base. Uh, when you have a huge volume of these feedback loops, uh, agents are able to operate for greater periods of time on more complex tasks without human intervention. So, we have like a product that we call missions, which I'll also touch on in a sec. But, missions is basically an extremely elaborate harness built around the concept of working on extremely difficult knowledge work problems that are validatable, right? And so, the quality of the output of these very long-running harnesses of advanced agents is directly proportional to the degree to which you can validate their work. And so, if you introduce the ability to validate at scale, then you introduce increasing autonomy to the org. So, what we'll look at is we have tools that help scan all of these things, but often times, uh, the change is not so simple. Uh, for I'd say maybe 30 to 40% of the low-hanging fruit, you click droid, please fix all of this and it'll go in and it'll fix it, right? But for the other 60% some of them involve workflow changes. Sometimes humans are not used to the degree of I would say like nitpickiness of these automated systems. And so you have to sort of be aware of the concerns, the the humans, you have to think about like the way that people are currently developing systems and say, "How do we introduce some of these more extreme validation strategies without interrupting the dev flow of the humans who are involved in the work?" Um and and I mentioned missions because really I think this is one of the more end game of the agent era at least, pre-software factory era. But the more end game of the agent era style harnesses where it's simply a long running harness that has almost no human intervention except for the planning stage, right? Where you go in and you say, "I would like to have this very bounded task. I know that I want to solve this task and here is what solving this task means. I will now basically push a lever of inference until the task is complete, right? And so that is actually unbelievably competent at solving problems where like is complete is verifiable. So if you can frame any problem as the set of verification uh systems that need to validate it, then you can solve that problem with AI today. Uh and we've seen this work on some pretty insane problem spaces like migrating, you know, 30, 40, 50 million plus line code bases uh fully autonomously, um working on advanced uh like deep learning strategies around biomed, health care uh sort of problems, uh financial institutions that optimize equity research where you can actually build models of different equities and sort of analyze and compare and and build sort of a system that can then back prop and or trade on top of the those equities. Um like it it's mind-blowing to me every day what I what I hear people are using with these tools, but it is not something that you can just download, install, and hit play, right? It does require agent readiness. So, if your code base isn't agent ready, you won't see any of the success of the most capable AI systems in the world today, right? So, this is why we want people to go in and help our customers and say, "Hey, look, you can solve this actually very difficult problem, but it is going to require a different form of investment than you were thinking. Less so solving the problem, more so preparing the environment for verification of the problem." And by the way, if you're familiar with how these models are actually trained, like this makes total sense, right? They they get dense reward when they get post trained on all these complex tasks. Models need dense reward. These verification signals form the basis of that reward that they use to keep them on track over a long-term goal-directed problem.