Loading lesson…
Applied LLM Systems: RAG, Agents & MCP / Retrieval-Augmented Generation
Why ground a model in retrieved sources.
Reviewed by Yuvaraj
A language model answers from its parameters, the patterns baked into its weights during training. That memory is vast but fixed, undated, and impossible to inspect. Ask about your company's refund policy or a paper published last week and the model has two options: it happens to have seen something close, or it improvises. Improvising confidently is what we call hallucination.
Retrieval-augmented generation (RAG) takes the pressure off memory. Before the model answers, the system retrieves relevant text from a source you control and places it directly in the prompt. The model's job shifts from "recall this fact" to "read these passages and answer from them."
Parametric memory has three weaknesses that no amount of scale fully removes:
RAG addresses all three at once by making the source material part of the input rather than something the model has to have memorized.
At its simplest, RAG is two steps wired together:
Answer from memory before revealing, retrieval practice is what builds durable recall.
What problem does retrieval-augmented generation primarily address?
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
The retrieval step is usually semantic: documents and the query are converted to embeddings and compared by meaning, not keywords. You will build up each piece of this over the next lessons, and assemble the whole pipeline in The RAG Pipeline.
Grounding, not magic
RAG does not make a model incapable of error, a wrong retrieval leads to a wrong-but-grounded answer. What it changes is where the answer comes from: visible passages you can cite and check, instead of opaque parameters. Good RAG systems return their citations for exactly this reason.
A common early question is whether to retrieve or to fine-tune. They solve different problems: fine-tuning changes how a model behaves; RAG changes what it knows at answer time.
Knowledge that changes often, must be cited, or is private to you is a retrieval problem, not a training one. You will compare them directly in a later course.
A fast-moving field
"RAG" now covers a spectrum, from a single retrieval step to reranking, query rewriting, hybrid keyword+vector search, and agentic retrieval that loops. The idea (ground generation in retrieved sources) is stable; the architectures change quickly. Treat specific pipelines as current practice, not settled fact.