Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Source
arxiv.org
Author
Patrick Lewis et al.
Date
Context
The original RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition → research gave a text-generating model an external source of information to consult. The 2020 paper combined a pretrained generator with a neural retriever that searched an indexed collection of Wikipedia passages. Retrieved passages then informed the generated response. The researchers compared using the same passages throughout an answer with allowing passage use to vary during generation. After fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition →, the combined systems improved question-answering results and produced more factual language than the tested generator-only baseline. The contribution was a trained connection between finding relevant text and composing an answer from it.
Terms in this piece · Glossary
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.