Sequence Processing Step by Step

An RNN reads tokens in order.

For:

I like tea

the model first reads I, then like, then tea. At each step it updates a hidden state.

h_1 = update(h_0, x_1)
h_2 = update(h_1, x_2)
h_3 = update(h_2, x_3)

The same update pattern repeats through the sequence.

Exercise

How many update steps are needed for the three-token sequence I like tea?