Word2Vec as History

Word2Vec is an important historical step in neural language representation.

Its training tasks were simple but revealing: use nearby words to learn vectors that are useful for predicting context. The details can vary, but the central lesson is stable: prediction pressure can shape word vectors.

Word2Vec made learned word geometry widely visible. Nearest neighbors, analogies, and vector arithmetic became common ways to inspect representations.

Why it belongs here

Modern LLM embeddings are not the same as old static word embeddings. But the historical idea prepares the mind: language tokens can be represented by learned vectors, and those vectors can contain structure induced by prediction.

Exercise

Does Word2Vec belong before or after neural token embeddings conceptually? Answer 1 for before, 2 for after.