Summary and Revision Notes

Core ideas

• Gates are learned controls over information flow.
• LSTMs separate cell state and hidden state.
• LSTM gates are often described as forget, input, and output gates.
• GRUs use update and reset gates.
• Gated recurrence improves memory but remains sequential.
• Attention later changes the route to earlier information.

Check yourself

• Can you explain what a gate near 0 or 1 means?
• Can you name the common LSTM gates?
• Can you explain why gated recurrence still motivates attention?