Vectors & Vector Spaces

In 2D or 3D you can literally draw the arrow. In the 768- or 1536-dimensional spaces used for embeddings you cannot picture it, but the same rules apply, which is exactly why the algebra matters more than the drawing.

Adding vectors and scaling them

Two operations define almost everything you do with vectors.

• Addition works component by component: line the vectors up and add matching slots. Geometrically, you place the tail of one arrow at the head of the other.
• Scalar multiplication multiplies every component by a single number (a scalar). A scalar of $2$ doubles the arrow's length; a scalar of $-1$ flips it to point the opposite way.

The magnitude (length) of a vector comes straight from Pythagoras:

$\lVert v \rVert = \sqrt{\sum_i v_i^2}$

Worked example: numbers you can check

Let $u = [2, 1]$ and $v = [1, 3]$.

1. 1Add component-wiseu + v = [2+1, 1+3] = [3, 4]. Only matching slots combine: first component with first, second with second.
2. 2Scale by a scalar2v = 2 × [1, 3] = [2, 6]. Every component is multiplied by 2, doubling the arrow's length while keeping its direction.
3. 3Measure the lengthThe sum [3, 4] has magnitude sqrt(3² + 4²) = sqrt(9 + 16) = sqrt(25) = 5, a clean whole number.

So $u + v = [3, 4]$, $2v = [2, 6]$, and $\lVert [3, 4] \rVert = \sqrt{3^2 + 4^2} = \sqrt{25} = 5$.

Vector spaces, basis, and dimension

A vector space is simply the full set of vectors you can build from a starting set using addition and scalar multiplication, and the result always stays inside the space (add two 2D vectors, get another 2D vector). A basis is a minimal set of vectors whose combinations reach every point in the space. In 2D the standard basis is $e_1 = [1, 0]$ and $e_2 = [0, 1]$; any vector such as $[3, 4]$ is just $3e_1 + 4e_2$. The dimension is the number of vectors in that basis, how many independent directions the space has.

Why AI turns everything into vectors

Machines do arithmetic, not meaning, so AI encodes data as vectors of numbers:

• Feature vectors: a house becomes $[\text{bedrooms}, \text{area}, \text{price}]$; an image becomes a long vector of pixel values.
• Embeddings: a word, sentence, or product is mapped to a dense vector so that closeness in space means similarity in meaning. "cat" and "kitten" land near each other; "cat" and "invoice" land far apart.

Once data lives in a vector space, similarity becomes distance, search becomes finding nearby arrows, and learning becomes nudging vectors around, all built from the two operations above.

Further reading

• 3Blue1Brown, Essence of Linear Algebra: the best visual intuition for vectors, spaces, and bases.
• MIT 18.06 Linear Algebra (Gilbert Strang): full university course with lectures, notes, and problem sets.
• Khan Academy, Vectors and spaces: step-by-step practice on the operations covered here.