Move activation scale without changing the real linear map

Use row activations X with shape tokens by input channels and weights W with shape input channels by output channels, so Y=XW. Let S be an invertible diagonal matrix of positive channel factors. Then X times inverse S and S times W produce the same Y in exact arithmetic. Scaling the wrong weight dimension is not an approximation of this identity; it is a different linear operation. The factors must be finite and nonzero, and their chosen numerical precision must avoid overflow or underflow.

Smoothing redistributes a range problem rather than eliminating it. Reducing a large activation channel can increase magnitudes in the corresponding weights and stretch their quantization groups. SmoothQuant selects such factors using activation and weight statistics to support weight-and-activation quantization. Its empirical quality results concern the models and settings tested in the paper. The algebra proves equivalence before rounding; it cannot prove that both newly quantized tensors, their product or the final language model will retain a particular accuracy.

To avoid an extra runtime scaling kernel, a compiler may fold inverse factors into the preceding operation. This is legal only when that graph transformation preserves every consumer. If one activation feeds three projections, each affected path must receive the corresponding compensation or retain an explicit scaling operation. Moving a diagonal factor across an arbitrary nonlinearity or residual addition is not generally valid. Biases, normalization parameters and shared tensors require separate treatment; a locally correct matrix identity does not automatically justify a whole-graph rewrite.

Keep the related algorithms distinct. AWQ uses activation information to choose channel rescaling that protects important weights during weight-only quantization; it does not perform GPTQ’s approximate second-order compensation of remaining weights. LLM.int8 instead separates selected outlier dimensions into a higher-precision contribution alongside a quantized contribution. Those are different numerical and kernel contracts. Combining their names does not combine their published results or make all intermediate tensors integer-only. Compare the actual exported representation and supported kernels, not an algorithm label alone.

First test the unquantized transformed graph against the original using controlled finite-precision tolerances. Then quantize and measure activation clipping, weight reconstruction and task-level changes independently. If the first comparison already fails badly, calibration cannot repair an incorrect rewrite. Select smoothing strength on calibration or development data and preserve an untouched final evaluation. Extreme factors can amplify scale rounding or require unsupported kernel ranges, so inspect representability as well as the apparent reduction of activation maxima.

For an axis-scaling analogy, Reset the matrix lab and set both a=0.25 and b=0. Predict determinant 0.25, and that the first basis vector maps to one quarter of its length while the second is unchanged. Play interpolates geometry; readouts remain those of the selected matrix. The compensating inverse scale four must be applied on paper to the matching weight channel. This restricted matrix family neither quantizes values nor verifies that a neural-network graph permits folding the factors.