Interpolation

Interpolation mixes estimates from different context lengths.

A trigram estimate may be specific but sparse. A bigram estimate may be less specific but more reliable. A unigram estimate is broadest.

Interpolation combines them:

mixed score =
  weight_3 * trigram score
  + weight_2 * bigram score
  + weight_1 * unigram score

The weights usually sum to 1.

Example

If the weights are:

trigram: 0.5
bigram: 0.3
unigram: 0.2

then all three levels contribute. The model does not have to choose only one context length.

Exercise

If interpolation weights are 0.5, 0.3, and 0.2, what is their sum?