Conclusion

RNNs introduced a clear neural way to process sequences: read one token, update a hidden state, repeat with shared parameters.

Unrolling shows the computation graph, and backpropagation through time trains the shared update rule. The strength is order-aware processing. The weakness is that long-range information must survive through many repeated steps.

Next we study that weakness directly.