Unify routed around OpenAI's 15 request per second cache ceiling

Unify, a company building AI tools for sales, describes spreading requests across cache keys so its agents can reuse shared instructions. Prompt caching avoids repeating computation on an unchanged prompt prefix, but a request must reach a machine holding the cached material, as OpenAI's documentation explains.

In LangChain's interview, the guest describes hashing user IDs into 16 groups and appending a random number from 1 to 30. The grouping lets users share cached system instructions and tool descriptions; the suffix distributes traffic across more keys. The intended balance is reuse without concentrating every request on one cache.

The numerical limit remains unresolved: the post and guest say 15 requests per second, while the documentation currently says traffic above 15 per minute can overflow to other machines. It also says GPT-5.6 and later handle cache routing automatically. Unify's account is a workload-specific design, not a universal recipe for current models.