Alignment Intuition

Attention can be read as soft alignment.

A current position asks: which other positions matter for this computation? Weights express the answer. A high weight means the model is reading strongly from that position.

In translation history, this idea helped models connect source words to target words. In language models, it helps each position read useful context.

Exercise

If attention weights over three positions are 0.2, 0.6, and 0.2, which weight is largest, multiplied by 10?