The Zero-Probability Problem

A plain count estimate assigns zero probability to a continuation that was never observed after a context.

Suppose after I like, the corpus contains:

tea: 2
code: 1
music: 0

The unsmoothed estimate gives:

P(music | I like) = 0

This says the continuation is impossible under the model. That is often too strong. The corpus may simply be incomplete.

Exercise

In the table above, what is the unsmoothed count of music after I like?