Polysemy and Context

Polysemy means one word has multiple related meanings.

For example, bank can mean a financial institution or the side of a river. A single static embedding has trouble with this because it gives one vector to the word type.

Modern language models use context to create contextual representations. The token bank in river bank and bank account can produce different internal vectors because the surrounding tokens differ.

This is one reason fixed word embeddings were not the end of the story.

Exercise

If a word has two common meanings, how many meanings are involved in the polysemy example?