Stable Softmax

Softmax turns logits into probabilities:

softmax_i = exp(logit_i) / sum(exp(logits))

The direct version can overflow when logits are large. The stable version subtracts the maximum logit first:

shifted = logits - max(logits)
softmax_i = exp(shifted_i) / sum(exp(shifted))

This does not change the probabilities. Adding or subtracting the same constant from all logits leaves softmax unchanged.

For logits [1000, 1001], subtract 1001:

[-1, 0]

Now exponentials are safe enough to compute.