Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
Models only know their training data and what's in the prompt. RAG closes the gap at request time: search a knowledge base for passages relevant to the question (usually with embeddings), put the best ones in the prompt, and have the model answer grounded in them.
It's the standard pattern for "chat with your docs," fresher-than-training-data answers, and citing sources. The quality ceiling is almost always the retrieval step — if the right passage isn't fetched, the best model in the world answers from vibes.
Suppose an assistant answers questions about a project handbook. Retrieval finds relevant passages; generation uses those passages to compose an answer. Ask the assistant to show the passage supporting its conclusion. A fluent answer alone cannot tell you whether the right passage was found.
Retrieval and answer quality are separate checks. If the source passage is irrelevant, improve the search. If the passage is correct but the answer misrepresents it, inspect the generation step. Changing the model may leave the retrieval problem untouched.
Background: LangChain: retrieval and generation.