LLM Primer

Epsilons in Denominators

An epsilon is a small positive value added to avoid unsafe division.

For example:

y = x / (norm + epsilon)

If norm is zero or extremely small, the denominator can cause unstable values. Epsilon makes the denominator safer.

0unsafe denominator+ epsilonsmall positive guard
Epsilon moves an unsafe zero or near-zero quantity away from the danger point.

Epsilon is not magic. If the main computation is wrong, epsilon will not make it right. It is a guard for edge cases where a denominator can become too small.

The size matters. Too tiny may not help. Too large may change the computation more than intended.

DL-C16-T06-001Exercise: Safe denominator

Let norm = 0 and epsilon = 0.001. What is norm + epsilon?

Compute it first, then check your number.

DL-C16-T06-002Exercise: Epsilon role

Enter 1 if epsilon can guard small denominators, or 2 if epsilon fixes every modeling problem.

Compute it first, then check your number.