Exercises

These exercises check the scale and normalization ideas from the chapter.

Exercise: Repeated scale

A signal starts at 2 and is multiplied by 3 across 3 layers. What is the final scale?

Your answer

Compute it first, then check your number.

Exercise: Vanishing scale

A signal starts at 16 and is multiplied by 0.5 across 4 layers. What is the final scale?

Your answer

Compute it first, then check your number.

Exercise: Normalize a feature

Let x = 25, mean = 15, and standard_deviation = 5. What is the normalized value?

Your answer

Compute it first, then check your number.

Exercise: Update size

Let learning_rate = 0.001 and gradient = 500. What is the update size before the minus sign?

Your answer

Compute it first, then check your number.

Exercise: ReLU activity

A ReLU layer has 40 units. If 30 units output zero, what fraction is zero?

Your answer

Compute it first, then check your number.

Exercise: Batch or layer normalization

Enter 1 for batch normalization, or 2 for layer normalization: this method uses statistics within each example's feature vector.

Your answer

Compute it first, then check your number.

Exercise: Scale warning

Enter 1 for likely healthy scale, or 2 for warning: loss is finite for two steps, then becomes NaN.

Your answer

Compute it first, then check your number.