Why Vector Databases?

You have already turned your documents into embeddings: millions of dense vectors, each a few hundred numbers that capture meaning. Now a user query arrives, you embed it too, and you need the handful of stored vectors closest to it, its nearest neighbors, in milliseconds, while new vectors keep streaming in. That single requirement is what a vector database exists to solve. It is not merely a place to keep vectors; it is a system built around the one query that dominates semantic search: "which of my millions of vectors are most similar to this one?"

Why brute force stops working

Exact k-nearest-neighbor search compares the query against every stored vector, computing a similarity for each. With $N$ vectors of dimension $d$, that is $O(N \cdot d)$ work per query. For a few thousand vectors this is trivial and you should just do it, a NumPy dot product or a single SQL scan is genuinely fine. The problem is linear growth: ten thousand vectors returns instantly, ten million does not.

A worked example: 10 million 768-dim vectors

N = 10,000,000 vectors, d = 768 dimensions, FP32 (4 bytes/number)

Brute force, per query:
  distance ops  = N * d       = 10^7 * 768    ≈ 7.7 x 10^9 multiply-adds
  memory read   = N * d * 4B  = 3.07 x 10^10  ≈ 30 GB scanned per query
  at ~30 GB/s memory bandwidth                ≈ 1 second PER QUERY

ANN index (e.g. HNSW), per query:
  candidates visited ≈ 5,000 vectors
  distance ops       = 5,000 * 768            ≈ 3.8 x 10^6 multiply-adds
  ~2000x fewer operations                     ≈ single-digit milliseconds