Conclusion

RNNs made sequence modeling neural and clear, but long sequences exposed hard problems. Gradients can vanish or explode, and a fixed hidden state can become a memory bottleneck.

Gated RNNs were an important answer. They added mechanisms for keeping, writing, and exposing information more deliberately.