Rounding Error
Rounding error is the difference between the exact mathematical result and the stored numerical result.
One rounding error is often tiny.
Many rounding errors can matter.
If an operation gives an exact result that is not representable, the computer stores a nearby representable result. The difference is rounding error.
Accumulation
If a computation adds thousands or millions of values, small errors may accumulate.
The order of operations can also change the final stored result.
This is one reason large sums may be computed with special algorithms or in higher precision.
Cancellation
Subtracting nearly equal numbers can remove leading digits that were shared.
The result may contain fewer reliable digits.
This is one reason stable formulas avoid unnecessary subtraction of similar large values.
For example, if two large values agree in many leading digits, their difference may be small. After subtraction, the shared leading digits vanish, and the remaining result may have fewer reliable digits.
The fix is not to fear subtraction. The habit is to notice when subtraction is asking small differences to survive after large nearly equal quantities have already been rounded.
Which operation is more likely to lose useful digits?
Enter 1 for subtracting two nearly equal large numbers, 2 for adding 1
and 2.
Compute it first, then check your number.
Hint
Solution
Subtracting nearly equal large numbers can lose useful digits. Enter 1. The
shared leading digits cancel, leaving the smaller difference more exposed to
earlier rounding error.
Can many tiny rounding errors matter after many repeated operations?
Answer it first, then check.
Hint
Think about long sums or many training steps.
Solution
Yes. One rounding error may be tiny, but many repeated operations can accumulate error.
Can changing the order of floating-point additions change the stored result?
Answer it first, then check.
Hint
Floating-point addition is rounded after operations.
Solution
Yes. Because intermediate results are rounded, changing the order of operations can change the final stored result.
Is subtracting nearly equal large values generally safer or riskier than adding small ordinary values?
Answer it first, then check.
Hint
Shared leading digits cancel.
Solution
It is riskier because cancellation can remove shared leading digits and leave fewer reliable digits in the result.
Enter 1 if subtracting two already-rounded, nearly equal large numbers can
leave a result with fewer reliable digits.
Compute it first, then check your number.
Hint
Ask what happens to the shared leading digits.
Solution
Enter 1. When nearly equal large values are subtracted, their common leading
digits disappear and any earlier rounding error can become more important.
Before Moving On
Rounding error is small locally, but computation can amplify it.