LLM Primer

Introduction

LSTMs and GRUs are gated recurrent networks.

They keep the step-by-step nature of RNNs but add gates that control what to keep, what to write, and what to expose. These gates made recurrent models more capable on longer sequences.

The point here is not to memorize every equation. The point is to understand why gates were introduced: ordinary RNNs overwrite memory too easily and send gradients through difficult paths.

gates control what information is kept, changed, and exposedkeepold memorywritenew signalreadhidden stategated paths help gradients and information survive across more steps
LSTMs and GRUs add gates so sequence models can manage memory instead of overwriting it blindly.

What this chapter covers

  • why gates help;
  • LSTM memory cell;
  • forget, input, and output gates;
  • GRU update and reset gates;
  • gated recurrence as an attention precursor.