prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
Why it matters
If you promise zero data retention downstream, implicit caching is the ambiguity that can break the promise. This separates what is physically stored, hashed KV tensors paged to NVMe, from the prompt text itself, and shows where providers disagree.
