LLM Primer

Chapter 13

LSTMs and GRUs

Gates, cell state, hidden state, forget/input/output gates, GRU update/reset gates, controlled information flow, and remaining limits.

What this chapter does

Gated recurrence tries to control what is remembered, forgotten, and exposed at each step.

Lessons

Read these in order.

Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.

  1. 01
    Introduction

    LSTMs and GRUs as gated recurrent models.

  2. 02
    Why Gates Help

    Gates as learned controls over memory flow.

  3. 03
    LSTM Memory Cell

    Separating cell state from hidden state for a steadier memory path.

  4. 04
    Forget, Input, and Output Gates

    The common gate names used to read an LSTM.

  5. 05
    GRU Update and Reset Gates

    A simpler gated recurrent unit with update and reset gates.

  6. 06
    Gated Recurrence as Attention Precursor

    Why gated memory helps but still leaves a sequential bottleneck.

Review and practice

Close the chapter deliberately.

Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.

Conclusion

What gated recurrence establishes before decoding and attention.

Summary and Revision Notes

A compact review of gates, LSTM state, GRU gates, and the remaining sequential bottleneck.

Exercises

Chapter-level practice for gate arithmetic and gated recurrence concepts.

Hints

Low-spoiler nudges for the Chapter 13 exercises.

Solutions

Explained solutions for the Chapter 13 exercises.

Before moving on

  • Explain the purpose of gates without memorizing every equation first.
  • Compare RNN, LSTM, and GRU memory behavior.
  • Understand why gates help but do not remove all sequence limits.

Where this leads

  • Decoding and Sampling
  • From Recurrence to Attention

Chapter progress