Introduction

Deep learning runs exact ideas on finite machines.

That means the model does not only have mathematical behavior. It also has numerical behavior. Values can overflow, underflow, round, become NaN, or become infinite.

The Mathematics path introduced numerical computation. This chapter applies that habit to deep learning: logits, softmax, losses, gradients, normalization, and mixed precision.

The goal is practical. When training fails numerically, you should know what to inspect and which stable rewrite to try first.