Numerical stability and mixed precision

Floating-point formats allocate bits to sign, exponent, and significand. Exponent range controls how large or small normal values can be; significand precision controls relative spacing within a range. Binary16 and bfloat16 make different tradeoffs, so replacing one with the other changes more than memory use. Subnormal handling and intermediate accumulation formats also matter. Check the documented execution path for the actual operation and hardware rather than assuming all arithmetic uses the input storage format.

For an ordinary correctly rounded operation whose exact result is normal and does not overflow, a relative rounding model is often useful. It is not a universal bound for underflow, overflow, cancellation-relative error, or an entire algorithm. Addition is not associative after rounding: a parallel reduction tree can change the result. A well-conditioned problem can still be implemented unstably, while an ill-conditioned problem may amplify tiny input errors even under a stable implementation.

Loss scaling multiplies the scalar loss before differentiation, making gradients larger before low-precision representation can discard tiny values. Gradients must be unscaled before operations whose meaning depends on their magnitude, such as ordinary gradient clipping, and before the intended optimizer update. If scaled gradients overflow, a dynamic scheme can reduce the scale and skip an unsafe update. Scaling is a range-management technique, not extra significand bits or a cure for a poor algorithm.

Higher-precision accumulation reduces some summation error, and higher-precision master weights can retain small updates that would round away in lower-precision storage. These techniques were central to early mixed-precision training recipes, but modern frameworks can use other format-specific strategies. Loss scaling is often less necessary for bfloat16's wider exponent range than for binary16; that does not imply every bfloat16 operation has adequate precision. Test the selected recipe rather than making a universal format claim.

Begin with a small deterministic case and a higher-precision reference where feasible. Compare absolute and relative error together, because relative error is unstable near zero. Inspect activation, gradient, and optimizer-state distributions by layer, along with nonfinite counts and skipped updates. A close output for one batch does not prove equivalent training: small systematic errors can accumulate, while harmless nondeterministic rounding may lead to different but similarly good optimization trajectories.

For unstable expressions, change the mathematics of evaluation. Subtracting a maximum before exponentiating in a softmax avoids unnecessary overflow without changing the exact normalized probabilities. Reductions may benefit from better ordering or compensated summation where cost permits. Measure end-to-end quality across seeds after local numerical checks. Report tolerance choices and hardware settings so another investigator can separate expected differences from a regression.