Summary and Revision Notes

Core ideas

• RNNs process sequences one token at a time.
• The hidden state carries a learned summary forward.
• The same recurrent parameters are shared across time.
• Unrolling draws repeated computation as a graph.
• BPTT trains the unrolled graph.
• Long gradient paths create difficulty for long dependencies.

Check yourself

• Can you count RNN update steps for a sequence?
• Can you explain why the hidden state is a summary, not a transcript?
• Can you explain why unrolling does not create separate parameter sets?