advanced24 min studyLesson 727 in Engineering

Prove accumulator bounds before shrinking operands

Track centered integer products, bias and intermediate widths instead of inferring safety from operand size.

On this page

What you’ll understand

  • Bound a centered integer dot product and its bias.
  • Compute a sufficient signed accumulator width with endpoint care.
  • Distinguish accumulator overflow from final output saturation.

Eight-bit operands do not imply an eight-bit dot product. Multiplication creates a larger integer and a long reduction adds many such products. Affine zero-point correction, integer bias and partial reductions can enlarge intermediate values further. A safe quantized kernel specifies every relevant width and proves that its arithmetic obeys the intended contract, rather than relying on a plausible-looking final dequantized value.

Center codes before reasoning about their magnitude

For affine operands, the mathematical integer accumulator sums products of qx minus zx and qw minus zw. The largest centered magnitude depends on both the stored range and the zero point. An unsigned eight-bit code can differ from its zero point by as much as 255; assuming the symmetric signed maximum 127 would understate that case. If weights have per-channel scales, the conversion back to real units also varies by output channel. Keep scale application separate from the integer magnitude proof.

A direct centered multiply and a kernel using expanded zero-point correction are algebraically equivalent over unbounded integers. Their intermediate requirements can differ because the expansion contains sums of raw codes and correction terms that later cancel. Check those terms and partial reductions in the implementation, not only the final centered sum. Bias expressed in accumulator units must be included too. A representable output after cancellation does not prove that an earlier narrow multiply or reduction was representable.

C=∑k=1K(qx,k−zx)(qw,k−zw)+bint,y^=sxswCC=\sum_{k=1}^{K}(q_{x,k}-z_x)(q_{w,k}-z_w)+b_{\rm int},\qquad \widehat y=s_xs_w C
C and integer bias are in product-code units. Positive operand scales s_x and s_w convert to output units; this expression assumes the bias has been represented in those accumulator units and does not include an additional output requantizer.↔ Scroll wide equations horizontally.

Prove a sufficient width, then inspect the kernel

If centered operand magnitudes are bounded by A and B, the triangle inequality gives an accumulator magnitude no larger than KAB plus the absolute integer bias. Requiring that bound to fit the positive signed maximum gives a conservative condition that protects either sign. The negative endpoint has one extra representable magnitude, but using it requires a more specific sign analysis. A power-of-two bound equal to the positive range limit is an easy place to make a one-bit error.

The ONNX MatMulInteger version 10 contract returns an int32 result and permits overflow at the 32-bit accumulation level; exporting that operator is not a proof that arbitrary dimensions are safe. Hardware can also use internal instructions with their own intermediate behavior. Inspect the selected backend and adversarially test long reductions, extreme centered codes and large bias. If a bound fails, widen, split the reduction with a proven wider combination, constrain ranges or change the representation. Do not silently substitute saturation for exact accumulation.

∣C∣≤KAB+∣bint∣≤2w−1−1|C|\le KAB+|b_{\rm int}|\le2^{w-1}-1
K is a positive integer reduction length, A and B are nonnegative centered-code magnitude bounds, and w is signed accumulator width in bits. The second inequality is a sufficient exact-integer condition; actual intermediate steps need their own compatible widths.↔ Scroll wide equations horizontally.

Output clipping cannot repair earlier overflow

A final requantizer typically rescales the wide accumulator, rounds and clips to a smaller output code range. That final saturation can be intentional and measurable. Overflow before it changes the accumulator itself, so clipping the corrupted value afterward does not recover the intended result. Keep counters or tests for the two failure modes separate. Test the complete conversion path, including multiplier approximation, shifts, tie rules and destination signedness; a correct accumulation proof alone does not prove bit-identical requantization.

In the modular lab, Reset and set both m=16 and a=7. Predict three Steps from zero as 7, 14 and 5, because the third addition wraps 21 modulo 16. A hypothetical saturating unsigned four-bit accumulator would instead give 7, 14 and 15. The instrument has neither signed dot products nor machine-specific overflow rules. Its explicit modulo arithmetic simply makes the loss of an intended mathematical sum visible; actual kernel bounds must be proved from their own centered operands and widths.

Count the bits in a long integer dot product

An exact dot product has K=1,024 terms, no bias and centered integer operands with absolute values at most 127. Find a sufficient signed width using the worst-case all-positive product sum. Also check whether a 16-bit accumulator can hold three maximal positive products.

  1. One maximal product is 127 times 127=16,129. The worst-case sum for 1,024 terms is 16,129 times 1,024=16,516,096, a dimensionless integer in product-code units.

  2. A signed 24-bit accumulator has positive maximum 8,388,607 and fails this bound. Signed 25-bit has maximum 16,777,215 and fits it. A signed 32-bit accumulator also fits, subject to compatible intermediate arithmetic and the stated no-bias premise.

  3. For the smaller test, two maximal products sum to 32,258 and fit signed 16-bit maximum 32,767. Three sum to 48,387 and do not. Even a short reduction therefore disproves the assumption that an arbitrary narrow accumulator is safe because each operand is eight bit.

The result

The long reduction needs at least 25 signed bits for this worst-case bound. Width proofs depend on centered ranges, reduction length and bias, not on the operand format’s name alone.

Explore the model

Interactive lab

Interactive lab · modular

Wraparound and saturation lose different information

Reset → set m=16 and a=7 → predict residues 7, 14, 5 after three Steps. Compare on paper with unsigned saturation at 15: 7, 14, 15.

Wraparound and saturation lose different informationAddition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Current readings: Residue r0: 0; Orbit period · m/gcd(m,a): 12; Distinct residues visited: 1 of 12. All parameters are adjustable with the labeled controls below.Repeated addition modulo 12: the current residue is 0, and the orbit period is 12.01234567891011+ 5mod 12An additive orbitn = 0rₙ = 0Period = 12Cycle starts at 0:0 → 5 → 10 → 3 →8 → 1 → 6 → 11 →4 → 9 → 2 → 7 → 0Exact residues · two additions per second
Additive orbitCurrent residue / next edge
Residue r0
0
Orbit period · m/gcd(m,a)
12
Distinct residues visited
1 of 12
t = 0 s

Addition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forThis is modular addition, not a signed accumulator or an integer inference kernel. Neither wrapped nor saturated values equal the exact sum 21.

Check your understanding

Why does a safe bound on the final centered dot product not always prove an expanded zero-point kernel safe?

Try it yourself

Bound K=4,096 products when each centered operand can have magnitude 128 and bias is zero. Does signed 27-bit storage suffice, or is another bit needed?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compute 4,096×128² exactly and compare with the positive endpoint 2²⁶−1 of signed 27-bit storage.

Compare with the solution

The magnitude bound is 4,096×16,384=67,108,864=2²⁶. Signed 27-bit maximum is only 67,108,863, one less. The worst-case positive sum therefore needs signed 28-bit storage, whose positive maximum is 134,217,727. The extra negative endpoint does not save a possible positive sum.

Take it with you

  • Centered ranges, bias and reduction length determine the accumulator bound.
  • Expanded correction arithmetic needs intermediate-width analysis.
  • Final requantization saturation does not repair earlier overflow.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.