Match training-time fake quantization to the deployed operator

A fake-quantization operation quantizes and reconstructs a value while keeping a floating-point tensor interface. That lets surrounding training code see the coarse forward values without requiring the final integer inference kernel at every step. The emulation must match deployed code ranges, scale granularity, zero points, rounding and clipping closely enough for the intended comparison. Training a symmetric per-channel operator and exporting an affine per-tensor operator changes the problem. Integer accumulation and backend-specific conversion may require additional verification beyond fake-quantized operands.

Observer state and fake quantization are independent controls in PyTorch 2.8 FakeQuantize. Statistics can keep updating while fake quantization is disabled, or be frozen while quantization remains active. A training schedule must say when each changes and which parameters or scales are learnable. Otherwise calibration drift can change the forward mapping throughout a supposedly stable evaluation. Export the actual frozen parameters and compare converted outputs with the final training-time emulation on held-out inputs before relying on task metrics.

Rounding makes this forward function piecewise constant, with zero classical derivative away from its jumps and no classical derivative at a jump. A straight-through estimator deliberately substitutes another derivative to permit updates. One illustrative choice passes gradient one inside a declared clipping interval and zero outside. That is an algorithmic surrogate, not calculus proving the rounded function has slope one. Different implementations choose different endpoint behavior and scale gradients; these choices belong in the training configuration and reproducibility record.

A floating-point latent weight can move without changing its reconstructed code until it crosses a rounding boundary. Conversely, one small latent update near a boundary can change the forward value by a whole step. Loss need not decrease at every update, and a successful smooth optimizer setting does not prove stability of a quantized network. PTQ methods such as GPTQ use a reconstruction strategy without turning into QAT merely because they solve an optimization problem. Compare their actual update rules rather than using “optimization” as the classification criterion.

Allocate data and compute for retention tests as well as the finetuning objective. QAT can adapt to a coarse representation but can also overfit its training mixture or forget previously useful behavior. Keep the final held-out set independent of observer tuning and hyperparameter selection. Measure converted model quality, memory and latency with the actual kernel shapes. A lower fake-quantized training loss neither guarantees backend agreement nor establishes a quality advantage over a cheaper PTQ configuration.

The optimization lab is a deliberately smooth comparison. Reset and set both step size η=0.5 and initial position x₀=2; predict the first Step reaches x=1 with f=0.5. Separately Reset and set both η=2.5 and x₀=2; predict x=-3 with f=4.5, above initial f=2. Step is one iteration here. This exact quadratic illustrates sensitivity to update size, but it contains no rounding, observer state or straight-through estimator and cannot recommend a QAT learning rate.