Noise control: tests then canary before a human is ever paged

merge that code review is still needed. So, how do you optimize for noise there? There could be enough signals that the uh AI agent uh could could think of uh improving and it could like arbitrarily send code reviews for them without knowing enough about your system. There are a few tips that I mentioned here could be very powerful in in making sure that the agent first has enough verifications done before looking for an human's attention. The integration test, unit test, the most basic functional test that your system needs to have should be run ahead of time by the coding agent itself so that it knows that as it is making code changes to optimize it, it is also not causing any production business logic to fail. So, as as you have good coverage and unit test, it gets more confidence that it gets more confidence that your that the change it is going to propose is actually right. Once it has made that functional correctness check, you still the our code coverage isn't typically ideal. There are still surprises in production and canary deployments could be very powerful in this case. If you provide enough automation around this, the canary deployment could be a prerequisite before the code review requires human attention. What is a canary? You typically have you typically have two machines, one containing your old code, another containing your new code, which could be the performance fixed code. And you send the same traffic to both of them over a period of let's say 10 minutes and try to compare the CPU usage across between them. The comparison report typically is is what you would want the AI agent to make decision on and judgment on whether it should open a code review for it. In this case, the observability report could contain standard infra things like how how much CPU reduction, how much latency reduced, is there an increase in error rate? If if that's the case, it should see that as a red signal to not proceed because it might have gotten the business logic incorrect in an attempt to optimize the code. And what I mentioned here is that this is not an AI problem. The observability, canary, verify logic, these are all standard checks that you need need to have in your system. And then final guardrail that you want to have is an engineer decision, like I mentioned, still very important, critical uh in the mix. Um Here's a mental model. Profiler gives the estimate, canary gives ground truth. Uh canary is the way you verify and evaluate that your uh suggested fixes actually uh going to improve and have some uh positive impact and no negative impact. An engineer makes the eventual decision.