Stochastic rounding is conditionally unbiased only under a precise rule

Let a and b be adjacent finite representable values with a<x<b. Choose b with probability equal to the distance from a to x divided by the gap b−a, and choose a with the remaining probability. Values closer to a therefore round down more often. Multiplying each destination by its probability gives an expected result exactly equal to x. A value already represented should remain unchanged under this rule. A midpoint happens to use equal probabilities, but moving away from the midpoint changes them.

The expectation should be conditional on the current exact operation result and the relevant prior computation history. Otherwise a correlation between random choices and previous errors can invalidate a claimed zero conditional mean. The stated construction assumes the finite neighbors exist and the chosen probabilities are implemented correctly. An overflow case is outside this bounded-neighbor derivation. A flush-to-zero policy, restricted random bit budget, or altered probability generator must be analyzed as the operation actually implemented rather than silently inheriting the ideal rule's guarantee.

The rounding error has two possible values, a−x and b−x. Squaring them and weighting by their probabilities gives variance equal to the product of the two distances from x to its neighbors. Thus unbiased does not mean exact on an individual operation or small compared with every application tolerance. Near a neighbor, one outcome is common and the other is rare; across many repeated operations, occasional larger stored moves can preserve the mean contribution that nearest rounding would always discard.

A conditional zero-mean error can support useful martingale-style analyses, but it does not automatically supply independent errors. Nor does it make every nonlinear function of the rounded values unbiased. Squaring a random rounded value adds its variance to the squared mean, for example. In a learning computation, activations, clipping, adaptive optimizer moments, and later parameter-dependent gradients can all change the relationship between local rounding and final output. Distinguish a bound or expectation for a specific numerical quantity from a claim about task accuracy or convergence.

A practical implementation needs random bits at a particular operation granularity. Reusing a random word across lanes, changing thread assignment, or selecting a different reduction can alter correlations even when each marginal probability appears plausible. A seed alone does not document those choices. For a controlled comparison, record the stochastic-rounding algorithm, random stream assignment, arithmetic modes, backend version, and accumulator types. Compare repeated seeds and report variability alongside mean behavior, because one favorable run is not evidence of a general improvement.

The sampling lab can display a separate idealized repeated-choice model. Reset, set both p=0.25 and n=100, predict the mean upward-choice fraction 0.25 and variance 0.001875, then Play or Step. Its bars are exact binomial probabilities for independent Bernoulli trials; playback highlights outcomes and does not draw random samples or execute floating-point operations. The displayed normal comparison is an approximation, and its Berry–Esseen number bounds a standardized CDF error rather than each bar. Interpret the model as repeated fixed-probability choices under independence, not as the distribution of an adaptive training trajectory.