Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

TL;DR

Combines a neural retriever with a generator so the model conditions its output on documents fetched from an external corpus rather than on parameters alone.

Why it matters

RAG named and formalized the pattern now behind most production LLM apps: retrieve relevant text, then generate grounded in it. It lets models cite sources and stay current without retraining.

Key ideas

• Retrieve passages with a learned dense retriever, then generate conditioned on them.
• Knowledge lives in an external index, so it can be updated without touching the model.
• Grounding in retrieved text reduces hallucination and enables citations.
• Retriever and generator can be trained together end to end.

Related concepts