Denoising weights: transform the objective, not just its target name

Fix the noisy input and VP coefficients. Reconstruct a clean prediction from a noise prediction, or derive both from a velocity prediction. The corresponding residuals are related by scalar Jacobians, so their squared norms acquire squared coefficient factors. Use the same per-coordinate mean or vector-sum convention in every expression. A target change can otherwise introduce an unnoticed factor of data dimension as well as a timestep-dependent scale. These identities compare tied predictions, not arbitrary independently trained networks.

The original DDPM work distinguishes a simplified denoising objective from terms associated with its variational formulation. The precise weighting depends on assumptions including reverse variance and endpoint treatment. Do not label every unweighted noise MSE an exact likelihood objective or interpret its numerical value as directly comparable to another parameterization's raw MSE. Log common reference errors or explicitly transformed quantities when comparing models. A lower training number can reflect changed weights rather than a better denoiser.

Here begin with a clean-prediction loss weighted by min(r,gamma), for positive coefficient SNR r and positive threshold gamma. Transforming that same loss gives noise weight min(r,gamma)/r and velocity weight min(r,gamma)/(1+r). The denominator in the velocity form follows from sigma squared being one over one plus r. Applying the clean weight directly to noise MSE would instead define another objective. Endpoint values require limiting or guarded rules consistent with the chosen schedule.

The Min-SNR paper motivates a timestep-task balancing strategy and reports empirical improvements in its experiments. Those results do not guarantee faster convergence, better fidelity, or a universal optimal gamma for every dataset, sampler, backbone, or conditional task. Diffusers 0.35.1's text-to-image example contains parameterization-specific weights that can be checked against this algebra. Other preconditioned, continuous-time, or flow objectives need their own derivation. A familiar option name is insufficient evidence that two training scripts optimize the same quantity.

Compute a per-example feature loss, multiply by its timestep weight, and apply the declared batch reduction. Dividing by batch size and dividing by the sum of weights are different estimators, especially when the sampled timestep mix changes. If timesteps are sampled nonuniformly, separate objective weights from proposal correction. Record weighted and unweighted losses by noise-level bin, gradient norms, and a fixed held-out evaluation. Keep training data, model size, compute budget, and sampling protocol comparable when attributing an improvement to weighting.

The function lab isolates a weighted scalar residual square. Reset, set a=0.50 and x=1, predict y=0.50, then Play or Step. Separately Reset, set a=2 and x=1, predict y=2, then Play or Step. Interpret a only as a chosen nonnegative loss weight and x as an error; the graph does not compute an SNR or train a denoiser. This controlled change shows why raw loss magnitude changes when weights change even if the underlying residual stays exactly one.