Embeddings
Efficient Estimation of Word Representations in Vector Space
L3 · AdvancedMikolov, Chen, Corrado, Dean · 2013 · ICLR Workshop 2013
TL;DR
Introduces word2vec, which learns dense word vectors from context so that geometric relationships in the vector space capture meaning and analogy.
Why it matters
word2vec made embeddings mainstream: it showed that meaning can live in the geometry of a vector space, the idea behind every semantic-search and retrieval system today.
Key ideas
- Predict a word from its context (CBOW) or the context from a word (skip-gram).
- Words used in similar contexts land near each other in the space.
- Vector arithmetic captures analogies (king − man + woman ≈ queen).
- Efficient training made large-vocabulary embeddings practical.
Related concepts
embedding