Unrolling Through Time

Unrolling draws the repeated RNN computation as a chain.

Instead of viewing the model as a loop, we draw one copy of the computation per time step:

x_1 -> RNN -> h_1 -> RNN -> h_2 -> RNN -> h_3

The drawn copies share parameters. Unrolling is a way to see the computation graph for a particular sequence length.

This matters because training needs a computation graph. Once the repeated steps are unrolled, backpropagation can follow the graph backward through time.

Exercise

If a sequence has 5 tokens, how many RNN steps appear in the unrolled graph?