Clip transcript
the upstream. The key questions are, can we find something we can measure? Can we apply changes incrementally? And can we get feedback on the quality of those changes? To illustrate that, I'm going to walk through a control loop that we use internally at Human Layer. Uh for our loop, we are incrementally migrating our RPC API to Effect. We adopted it for some of our race-prone code. We like it, so we're adopting it across the rest of our code base. If you've never seen Effect's code before, the code on the right is just the kind of trivial procedure on the left rewritten in Effect. Uh the syntax is really weird. We're psychos. We really like it. It's not for everybody. That's okay. Uh this isn't a talk about Effect, so we'll keep moving. Ooh, clicker's not working. Cool. So, step one, we have to build our sensor to find unmigrated procedures. We can have an agent do this or we could use grep or rip grep. But, instead we're going to use AST grep cuz it's really powerful. It's a great tool to have in your toolbox for building loops. It's language-agnostic. It's out of band from your TypeScript config or ESLint rules, which if you're a TypeScript developer, you have watched Claude disable those with inline comments. Uh but, so we can just write a simple rule that finds unmigrated procedures uh based on the pattern above. We Over time, we can even layer on more rules that describe other patterns we want to get rid of with granular include and exclude paths. If you have a multilingual monorepo like we do, uh it'll work for any language you could possibly imagine. And we can just scan our codebase and it'll produce a long list of violations. Uh way too long, in fact. It'll give you about 50 keys per violation, so we're just going to filter it down to four and we're going to sort it deterministically. Why are we doing that? At the beginning, I said this was going to be practical. And so, we're going to step outside of our control loop paradigm for a second because before we start incrementally migrating procedures one at a time, we need to enforce that all new procedures are using effect, right? So, we're going to run a full scan once on main, sort all the violations deterministically, and track it in our version control. And then on every new PR, we can see if it the branch added any unmigrated procedures, right? So, this is our control loop and our system is undergoing disturbances. In this case, uh all of our teammates shipping Claude's law. And this is how we make sure that they're not undoing our loop's work. This doesn't map directly to a part of the control loop, but we can kind of like squint at it a little bit and call it a disturbance dampener. So, now that we've stopped the bleeding, we can actually design our controller. For a simple controller, we could just deterministically pick the first violation from the list. You can use bash and jq or we could get a little cleverer and use AST grep to find the smallest unmigrated procedure and always pick the smallest one to reduce the risk. Uh we could have an agent make the decision if we really want to. I don't think you should ever send an agent to do deterministic code's job, but you certainly can. In fact, depending on the complexity, we could have the agent pick the procedure to migrate and just do it at the same time like we just talked about. But we can make this even more powerful, right? Because we're not just migrating to a factor for the sake of it. We're doing it because it's helpful for handling errors and for helping us instrument our code better. And so what we could do if we want to get really clever is we can look at our telemetry and figure out which procedures have the most errors or the least instrumentation or has a gap in our APM, right? And when we send a control signal to our actuator agent, we can include not just the procedure to migrate, but also all the data about the things that we're trying to fix with this migration so that the actuator agent can actually make the code better instead of just doing a one-to-one migration.