Mixed Precision Preview
Mixed precision uses more than one dtype during training.
The goal is usually speed and memory efficiency while keeping enough stability. For example, some values may be stored or multiplied in lower precision while sensitive accumulations stay in higher precision.
This is a preview, not a framework recipe.
The important idea is:
use lower precision where it is safe
use higher precision where stability needs it
Mixed precision can require additional techniques such as loss scaling. Loss scaling helps prevent gradients from underflowing when represented in lower precision.
You do not need to implement mixed precision yet. You need to understand why larger models care about dtype and why stable rewrites remain important.
Enter 1 if mixed precision can use more than one dtype, or 2 if it requires every value to use the same dtype.
Compute it first, then check your number.
Enter 1 if lower precision can save memory but needs stability care, or 2 if lower precision removes all numerical risks.
Compute it first, then check your number.