Dropout
Dropout is a training-time perturbation.
During training, it temporarily removes some activations. The network must learn to work without relying on one exact set of hidden units every time.
At evaluation time, dropout is usually turned off. The exact scaling convention depends on implementation, but the concept is stable: train with random missing pieces, evaluate with the full network in a matched scale.
Dropout is not always the right tool. It can help reduce co-adaptation in some models, but it can also slow training or be unnecessary when other regularization is enough.
For now, understand its role: dropout makes training less deterministic so the model cannot depend too tightly on fragile hidden-unit combinations.
Enter 1 if dropout is mainly a training-time perturbation, or 2 if it is mainly an evaluation-time perturbation.
Compute it first, then check your number.
A layer has 20 activations. Dropout removes 5 during one training step. How many remain active?
Compute it first, then check your number.