Dot Product & Similarity

That is literally all the machine does, a tight multiply-and-add loop over the components, which is why hardware (CPUs and GPUs) is built to run it billions of times per second.

The geometric form

The exact same number can be written using lengths and the angle $\theta$ between the two vectors:

$a\cdot b=\lVert a\rVert\,\lVert b\rVert\cos\theta$

Here $\lVert a\rVert=\sqrt{a\cdot a}$ is the length (norm) of $a$. These two formulas are equal: the algebraic form is how you compute the dot product, and the geometric form is what it means.

Sign: are they pointing the same way?

Because lengths are never negative, the sign of the dot product is entirely determined by $\cos\theta$:

This is why the raw dot product works as a similarity score: bigger and more positive means "more aligned."

Projecting one vector onto another

Projection answers a related question: how much of $a$ lies along the direction of $b$? The scalar projection (the length of $a$'s shadow on $b$) is

$\frac{a\cdot b}{\lVert b\rVert},$

and the vector projection puts that shadow back as a vector pointing along $b$:

$\operatorname{proj}_b a=\frac{a\cdot b}{b\cdot b}\,b.$

Projection is the geometric engine behind least-squares regression, Gram–Schmidt orthogonalization, and dropping data onto principal components in PCA.

Cosine similarity

Solve the geometric form for the angle term and you get cosine similarity:

$\cos\theta=\frac{a\cdot b}{\lVert a\rVert\,\lVert b\rVert}.$

Dividing by both lengths cancels magnitude and leaves only direction, giving a score from $-1$ to $1$. This is the backbone of semantic search: an embedding model turns text into vectors whose direction encodes meaning, so two passages about the same topic point almost the same way and score near $1$. Vector databases rank retrieved results by exactly this number.

1. 1Multiply and sumPair up matching components, multiply each pair, then add them into one number.
2. 2Measure each lengthCompute the two norms: the square root of each vector dotted with itself.
3. 3Divide out lengthCosine similarity = (a·b) / (||a|| · ||b||); dividing by both lengths removes magnitude and leaves pure direction.
4. 4Interpret the scoreRead it on a scale from -1 (opposite) through 0 (unrelated) to 1 (identical direction).

Worked example

Take $a=(2,3)$ and $b=(4,1)$.

• Dot product: $a\cdot b = 2\cdot4 + 3\cdot1 = 8+3 = 11$.
• Norms: $\lVert a\rVert=\sqrt{2^2+3^2}=\sqrt{13}\approx3.61$ and $\lVert b\rVert=\sqrt{4^2+1^2}=\sqrt{17}\approx4.12$.
• Cosine similarity: $\dfrac{11}{\sqrt{13}\,\sqrt{17}}=\dfrac{11}{\sqrt{221}}\approx0.74$.

Since $11$ is positive and the cosine is close to $1$ (the angle is about $42^\circ$), the two vectors are fairly similar in direction.

import numpy as np

a = np.array([2, 3])
b = np.array([4, 1])

dot = a @ b                                    # 11
cos_sim = dot / (np.linalg.norm(a) * np.linalg.norm(b))
print(dot, round(float(cos_sim), 3))           # 11 0.74

Further reading

• 3Blue1Brown, Essence of Linear Algebra (dot products, chapter 9), the definitive visual intuition for why the two forms of the dot product agree.
• MIT 18.06 Linear Algebra, Gilbert Strang (MIT OpenCourseWare), rigorous lectures covering dot products, projections, and orthogonality.
• scikit-learn: cosine_similarity, the practical API used across real ML retrieval pipelines.