Conclusion

LSTMs and GRUs improved recurrence by adding gates. Gates let the model control what to keep, write, reset, and expose.

This made recurrent models more effective for long sequences, but it did not remove the sequential bottleneck. That limitation prepares the next step in the story: decoding and then the move from recurrence to attention.