Rounding Error

Rounding error is the difference between the exact mathematical result and the stored numerical result.

One rounding error is often tiny.

Many rounding errors can matter.

If an operation gives an exact result that is not representable, the computer stores a nearby representable result. The difference is rounding error.

Accumulation

If a computation adds thousands or millions of values, small errors may accumulate.

The order of operations can also change the final stored result.

This is one reason large sums may be computed with special algorithms or in higher precision.

Cancellation

Subtracting nearly equal numbers can remove leading digits that were shared.

The result may contain fewer reliable digits.

This is one reason stable formulas avoid unnecessary subtraction of similar large values.

For example, if two large values agree in many leading digits, their difference may be small. After subtraction, the shared leading digits vanish, and the remaining result may have fewer reliable digits.

The fix is not to fear subtraction. The habit is to notice when subtraction is asking small differences to survive after large nearly equal quantities have already been rounded.

MATH-C08-T03-001Exercise: Spot cancellation

Which operation is more likely to lose useful digits?

Enter 1 for subtracting two nearly equal large numbers, 2 for adding 1 and 2.

Compute it first, then check your number.

Hint
Think about digits that cancel out.
Solution

Subtracting nearly equal large numbers can lose useful digits. Enter 1. The shared leading digits cancel, leaving the smaller difference more exposed to earlier rounding error.

MATH-C08-T03-002Exercise: Rounding can accumulate

Can many tiny rounding errors matter after many repeated operations?

Answer it first, then check.

Hint

Think about long sums or many training steps.

Solution

Yes. One rounding error may be tiny, but many repeated operations can accumulate error.

MATH-C08-T03-003Exercise: Order can matter

Can changing the order of floating-point additions change the stored result?

Answer it first, then check.

Hint

Floating-point addition is rounded after operations.

Solution

Yes. Because intermediate results are rounded, changing the order of operations can change the final stored result.

MATH-C08-T03-004Exercise: Avoid the cancellation trap

Is subtracting nearly equal large values generally safer or riskier than adding small ordinary values?

Answer it first, then check.

Hint

Shared leading digits cancel.

Solution

It is riskier because cancellation can remove shared leading digits and leave fewer reliable digits in the result.

MATH-C08-T03-005Exercise: Notice dangerous subtraction

Enter 1 if subtracting two already-rounded, nearly equal large numbers can leave a result with fewer reliable digits.

Compute it first, then check your number.

Hint

Ask what happens to the shared leading digits.

Solution

Enter 1. When nearly equal large values are subtracted, their common leading digits disappear and any earlier rounding error can become more important.

Before Moving On

Rounding error is small locally, but computation can amplify it.