Diffusion and score models

Let beta_t define a variance schedule, alpha_t equal one minus beta_t, and alpha-bar_t be the product of alphas up to time t. The Gaussian forward process has a closed-form marginal at any selected time. This lets training draw a clean example and independent noise, then construct one noisy example directly. It need not simulate every previous corruption step. The schedule changes signal-to-noise ratios across time and therefore which denoising problems the model sees.

A neural network receives the noisy input and a time or noise-level embedding; conditional models also receive text, labels, or another conditioning signal. A common simplified objective regresses the sampled noise with squared error. Alternative parameterizations predict clean data or velocity-like combinations. These can be transformed under a known schedule but differ in numerical behavior and loss weighting. Record the exact target and weighting rather than describing every system simply as predicting the image.

The score is the gradient of log density with respect to the noisy input. For a Gaussian perturbation of a fixed clean example, its conditional score is a scaled negative noise residual. The marginal score averages the appropriate conditional behavior over possible clean examples given the noisy observation. A trained noise predictor estimates that conditional expectation under its loss assumptions; it is not handed the original clean example at generation time.

For a matching variance-preserving parameterization, an estimated score can be obtained by scaling a noise prediction by minus one over the noise standard deviation. Reverse stochastic differential equations and probability-flow ordinary differential equations then provide different sampling descriptions. They require appropriate coefficients and numerical integration. A score arrow is a direction in data space, not a guarantee that one unrestricted step reaches a high-quality example or increases likelihood monotonically.

Each sampler step may require one or more network evaluations. Conditional guidance can require additional conditional and unconditional evaluations unless combined or otherwise optimized. A model with fewer parameters can therefore be slower to sample than a larger one using fewer steps. Report number of function evaluations, batch size, resolution, precision, guidance settings, and measured latency. Latent diffusion moves the denoising process into a learned compressed space and adds encoder-decoder assumptions and costs.

Evaluate fidelity, diversity, conditioning adherence, and failure cases separately. Aggressive guidance may improve one metric while reducing variety or introducing artifacts. Changing a noise schedule or using an accelerated sampler is a method change that needs validation. Neither a low denoising loss nor visually attractive selected samples alone establishes coverage of the target distribution. Keep prompts, seeds, sample selection, and evaluation sets available so a comparison can be reproduced without cherry-picking.