Simple Gates

A gate controls how much signal passes.

A simple gate value between 0 and 1 can mix an old signal and a candidate signal:

output = gate * candidate + (1 - gate) * old

A gate learns how much of a signal to pass or keep.

If gate = 1, the output uses the candidate. If gate = 0, it keeps the old signal. Values between 0 and 1 mix them.

For example:

old = 10
candidate = 4
gate = 0.25

output = 0.25 * 4 + 0.75 * 10 = 8.5

Gates appear in many architectures because they let the network learn when to update, keep, or blend information.

DL-C15-T05-001Exercise: Gate mix

Let old = 6, candidate = 2, and gate = 0.5. What is the output?

Your answer

Compute it first, then check your number.

DL-C15-T05-002Exercise: Gate zero

If gate = 0, old = 9, and candidate = 3, what is the output?

Your answer

Compute it first, then check your number.