Summary and Revision Notes

Core ideas

• An embedding matrix has one row per token id.
• Token ids are discrete; embeddings are learned continuous vectors.
• Similar contexts can produce related embedding geometry.
• Word2Vec is an important historical precursor.
• Nearest neighbors are useful clues, not proofs of meaning.
• Polysemy motivates contextual representations.

Check yourself

• Can you compute embedding matrix rows from vocabulary size?
• Can you explain why token ids alone are not meaningful?
• Can you explain why one static vector may fail for a word with multiple uses?