← All IntelClip / EducationDesigning for S3 latency: why P99 (not P50) drives architecture
From Fireside Chat: Simon Eskildsen on Building Turbopuffer · ≈24:36
“The P99 on a uh 256 or 512 kilobyte object on S3 um is around 200 milliseconds.”
“When you're designing a system, you want to optimize for the P99.”
“So, it's like if you're navigating a tree on S3, right? It's like, okay, you get the upper layer of the tree 200 milliseconds. You get like another layer of the tree 200 milliseconds. You get a bunch of leaves of the tree in 200 millonds.”
What’s in it
- War story on hitting S3 latency walls building a fast index
- Explains why P99, not P50, drives distributed system design
- Shows how tree-style S3 lookups multiply round-trip latency
Clip transcript
I spent the summer of of 23 just hammering my head against the wall trying to find an approach where I could get the latency that I wanted. Um, >> because the problem with S3 is it has really good durability, but latency we're talking hundreds of milliseconds, right? >> Yes. The P99 on a uh 256 or 512 kilobyte object on S3 um is around 200 milliseconds. Um, >> and and you're saying P99 because like when you're talking large scale, you want to care about the P99, right? >> Yeah. I think when you're >> That's why we're not talking about P50. >> When you're designing a system, you want to optimize for the P99. And especially because when you're designing a system on on S3, generally in every roundtrip, you're not doing one request. You're often doing lots of requests, right? >> You're going to hit the P99 real quick. >> Exactly. So, it's like if you're navigating a tree on S3, right? It's like, okay, you get the upper layer of the tree 200 milliseconds. You get like another layer of the tree 200 milliseconds. You get a bunch of leaves of the tree in 200 millonds. So in aggregate you have like you want to look at the P99 probably even the P999 to design the system properly because you will need to minimize the number of round trips that you had to make. So, I just sat and sketched that out um and tried a bunch of different approaches and then and then finally in in in July of 23, I I I got something end to end that seemed to work and then rewrote it probably twice and then released it in
Comments
Checking sign-in…
Loading comments…