Word2Vec
Word2Vec (Google, 2013) learns word vectors from their contexts, via the CBOW and Skip-gram architectures. We explain the famous vector arithmetic and its caveat, why it yields static embeddings that cannot handle polysemy, and where it sits against the contextual embeddings of 2018.
Word2Vec is a method for learning vector representations of words, called embeddings, from large amounts of text. It was presented in 2013 by a Google team led by Tomas Mikolov. Its idea is to place each word in a vector space so that words with similar meanings or uses end up close together, learning those relationships from the contexts in which they appear.
How it works
Word2Vec proposes two shallow neural network architectures. In CBOW the model predicts a word from those in its context; in Skip-gram it does the reverse, predicting the context from the word. A second paper that same year introduced the training techniques now taken as standard, such as negative sampling. The most cited result is vector arithmetic: the vector for “king” minus “man” plus “woman” lands near “queen.” An honest caveat is in order: that effect depends in part on the evaluation protocol, which excludes the input words when searching for the result; without that exclusion, the nearest neighbor is usually “king” itself.
Where it sits historically
Word2Vec produces static embeddings: a single vector per word, regardless of the sentence. That is its central limitation: it cannot handle polysemy, so “bank” (riverside), “bank” (finance) and other senses share the same vector. It is a 2013 method and should not be presented as the state of the art. In 2014 GloVe appeared, with a different approach based on global co-occurrence, and in 2016 fastText, which adds subword information and handles out-of-vocabulary words. The deeper shift came in 2018 with contextual embeddings—ELMo and BERT—which assign a different vector to the same word depending on its context, and later with the large language models.
What it is still good for
Word2Vec is no longer the state of the art, but it remains useful for its simplicity and low cost: as a teaching starting point, in low-resource or GPU-free settings, and in studies of bias or semantic change, where its word-by-word transparency is an advantage. For any task that depends on context or ambiguity, however, contextual models surpass it.
