← All IntelClip / AI AgentsThe manual profiling loop, step by step, and why nobody runs it
From AI Agents for Performance: Ship Faster, Pay Less — Rajat Shah, Netflix · ≈2:31
Walks the full human workflow — trigger profiling on one instance, open a visualizer, hunt the flame graph for many minutes, grep the repo, ship a review — to locate exactly where the 20-minute bottleneck sits.
What’s in it
- Walks the full human workflow — trigger profiling on one instance, open a visualizer, hunt the flame graph for many minutes, grep the repo, ship a review — to locate exactly where the 20-minute bottleneck sits.
Clip transcript
Typically, a a human would trigger profiling on a single production instance of a fleet of production instances. You would go and download it, potentially open it in a visualizer. The raw data that you download typically is not great to look at. It It could be, for example, a JSON structure data of the call stack and where the CPU is spent. So, using a visualizer helps you at least see and visualize the call stack and CPU time of various um method in your services better. Once you have that visualizer open, you pretty much end up spending a lot of time in uh like just looking at and and finding in this in this treasure hunt on on the path on the potential places where you could improve the code to to make it more performant. This takes a lot of time in order to even learn how to look at it. Um And there's a learning curve to curve curve to it. And this is where the real bottleneck ends up being. You end up you end up having to spend straight many, many minutes to uh to identify the bottlenecks. Once you've identified some code paths and some packages that that are spending significant CPU cycles, you you would end up searching it in your code bases in your code repos and see if it has a potential to improvement. Hopefully, you you hit you have luckier and you find a root cause and you produce a code review out. You merge it and you get some performance wins. And then you repeat all of this again. You see the problem, right? This is a very manual effort and very tedious effort to get right and this ends up being a bottleneck if you were to do it across many of your code bases and code paths and that's why this is done very rarely. People typically end up looking at profiling data only when something is going wrong at 2:00 a.m. and somebody needs to fix a problem because your CPU is unbearable.
Recommended reads
- clipThe 9-step bug pipeline and where humans actually belongAI Engineer
articleLoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding AgentHan Li, Zhemin Fang, Rili Feng, Yingqi Zhao, Jiaheng Liu, Pengfei Gao, He Ye, Dayi Lin, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang- clipThe real speedup: incident archaeology, not typingAI Engineer
Comments
Sign in to comment.
Loading comments…