Why Gates Help

A gate is a learned value that controls flow.

If a gate is near 1, information passes strongly. If it is near 0, information is blocked or reduced.

This gives a recurrent model a way to preserve useful information instead of blindly overwriting the hidden state at every step.

Simple arithmetic

If old memory is 10 and a keep gate is 0.8, the kept amount is:

0.8 * 10 = 8

The gate is not a human rule. It is produced by learned parameters.

Exercise

If old memory is 10 and the keep gate is 0.6, what amount is kept?