Chapter 11
Recurrent Neural Networks
Step-by-step sequence processing, hidden state, shared parameters across time, unrolling, and backpropagation through time.
What this chapter does
Recurrence is the first serious neural sequence model: one update rule is reused as a hidden state moves through the sequence.
Lessons
Read these in order.
Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.
- 01Introduction
RNNs as step-by-step sequence models with hidden state.
- 02Sequence Processing Step by Step
Reading tokens in order and updating hidden state at each step.
- 03Hidden State
The learned running summary carried through a sequence.
- 04Shared Parameters Across Time
One recurrent update rule reused at every sequence position.
- 05Unrolling Through Time
Drawing repeated recurrent computation as a sequence graph.
- 06Backpropagation Through Time
Training an RNN by backpropagating through the unrolled graph.
Review and practice
Close the chapter deliberately.
Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.
What RNNs establish before long-range dependency problems.
Summary and Revision NotesA compact review of hidden state, shared parameters, unrolling, and BPTT.
ExercisesChapter-level practice for RNN sequence steps and hidden-state reasoning.
HintsLow-spoiler nudges for the Chapter 11 exercises.
SolutionsExplained solutions for the Chapter 11 exercises.
Before moving on
- Trace how an RNN carries information from earlier tokens.
- Explain shared parameters across time.
- Recognize BPTT as backpropagation through an unrolled graph.
Where this leads
- Long-Range Dependency Problems
- LSTMs and GRUs