Loading lesson…
Applied LLM Systems: RAG, Agents & MCP / Retrieval-Augmented Generation
Exact vs. approximate nearest neighbours.
Reviewed by Yuvaraj
Embedding two texts and comparing them is easy. Doing it across millions of stored vectors, fast enough to answer while a user waits, is the real engineering problem. That is what a vector database is for, and the operation at its heart is nearest-neighbour search.
Store an embedding for every passage in your collection. At query time, embed the question and find the stored vectors closest to it by cosine similarity. Return the top k. Those are your retrieved passages.
Done exactly, this is brute force: compare the query against every stored vector. It is perfectly accurate and, past a few hundred thousand vectors, too slow, every query touches the entire collection.
The fix is to give up a little accuracy for a lot of speed. Approximate nearest-neighbour (ANN) indexes organize vectors ahead of time so a query only has to look at a promising fraction of them.
Answer from memory before revealing, retrieval practice is what builds durable recall.
Why do vector databases often use approximate nearest-neighbour search?
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
Two common ANN families:
Say the true answer is the top 10 neighbours a brute-force scan would return, your gold set. An IVF index that probes only 2 of 50 clusters might return 8 of those 10. Its recall is 8/10 = 0.8, while it scanned roughly 1/25 of the vectors. Probe more clusters and recall climbs toward 1.0, but the query slows down. That dial is the whole game.
Try it in the Vector Search Simulator
The linked lab embeds a small corpus and lets you switch between exact (brute force) and approximate (IVF-style) search, adjust k and how many clusters to probe, and watch the recall and the number of vectors actually scanned change. It is a teaching illustration of the IVF idea, not a production HNSW index.
More probing → higher recall, slower queries. Less probing → faster queries, some true neighbours missed.
Choosing where to sit on that curve, and measuring recall against an exact baseline, is a core skill in building retrieval systems.
Real systems rarely stop at vector similarity. They add metadata filtering (restrict to a customer, a date range), hybrid search (combine keyword and vector scores), and reranking (a stronger model re-scores the top candidates). You will see where these fit in the next lesson.