Genai Gateway

Uber Engineering describes a shared gateway that puts model access and data handling in one place. Teams send requests through a common interface, while the gateway connects to external providers or internally hosted models. Its privacy layer replaces detected personal information with placeholders before sending requests outside the company, then restores the original information in returned responses. The July 2024 engineering account explains how that arrangement centralizes controls that individual teams would otherwise implement separately.

The same account identifies a tradeoff: redaction adds processing time and can affect answer quality. Centralizing the mechanism makes its behavior a shared engineering concern.