Bigram Models

A bigram model uses the previous token as context.

For the sequence:

I like tea
I like code

the token like follows I twice. The tokens tea and code each follow like once.

The bigram estimate is:

P(next = tea | previous = like) = 1 / 2
P(next = code | previous = like) = 1 / 2

The model has learned one-step context. It still cannot see more than the previous token.

Exercise

In the two short sentences above, how many times does like follow I?