Forget, Input, and Output Gates

An LSTM is often described with three gates.

The forget gate controls how much old cell state remains. The input gate controls how much new candidate information is written. The output gate controls how much of the cell state is exposed as hidden state.

In plain language:

forget: what old memory remains?
input: what new memory is written?
output: what memory is shown now?

These gates are learned functions of the current input and previous state.

Exercise

How many gates are named on this page?