Conclusion

Initialization, scale, and normalization make training less fragile.

The main lesson is practical: before changing the architecture, inspect the numbers. Activations, gradients, parameters, and losses should be finite and in useful ranges.

You have now seen why:

• initialization sets the starting scale
• activation scale affects what later layers receive
• gradient scale affects update size
• input normalization reduces arbitrary unit effects
• layer, batch, and RMS normalization help keep intermediate values trainable

The next chapter separates fitting from generalizing. A model can train smoothly and still learn patterns that do not hold on new data.