chunking — Splitting documents into passages small enough to embed and retrieve individually — the step that quietly determines whether retrieval works at all.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
reranking — A second pass that re-scores retrieved candidates by reading each one against the query, fixing the ordering that fast vector search got approximately right.
prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
Why it matters
Prepending chunk-specific context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → before embedding, combined with lexical BM25 and rerankingA second pass that re-scores retrieved candidates by reading each one against the query, fixing the ordering that fast vector search got approximately right.Full definition →, cut retrieval failures by up to 67% in Anthropic's tests. It is a concrete preprocessing change most existing RAG pipelines can adopt directly.
