Unigram Models
A unigram model ignores context. It estimates the probability of each token from its overall frequency.
Suppose a corpus has these tokens:
red blue red green red
The counts are:
| token | count |
|---|---|
| red | 3 |
| blue | 1 |
| green | 1 |
There are 5 tokens total, so:
P(red) = 3 / 5
P(blue) = 1 / 5
P(green) = 1 / 5
This model knows which tokens are common. It does not know what tends to follow what.
Exercise
In the corpus above, what is the count of red?
Compute it first, then check your number.