research30 min studyLesson 781 in Engineering

Reverse SDE and probability-flow ODE

Separate stochastic reverse dynamics from deterministic probability transport, with explicit time direction, score assumptions, and numerical limits.

On this page

What you’ll understand

  • Write reverse-time dynamics for a scalar state-independent diffusion coefficient.
  • Explain when a probability-flow ODE shares marginal densities with an SDE.
  • Distinguish continuous-time identities from finite-step learned-model sampling.

A diffusion serving system does not simply run training noise backward. It executes a particular numerical process using a learned field, an initial distribution, a time convention, and a discretization. Reverse stochastic dynamics and a probability-flow ordinary differential equation can be related at the level of marginal densities under specific assumptions, while their individual paths and finite-step outputs remain different. Keep those statements separate before comparing speed or quality.

Reverse the process with the correct score and time sign

Consider a forward Itô process with drift f(x,t), scalar diffusion coefficient g(t) independent of state, and smooth marginal density p_t. Its score is the state gradient of log p_t, not a predicted clean sample unless the required parameterization conversion has been applied. The reverse-time formula uses the forward drift minus g squared times that score and integrates from terminal time T toward zero. In this convention dt is negative and backward Wiener increments have variance equal to the magnitude of dt. Reversing the sign twice changes the dynamics.

An equivalent increasing computational coordinate τ=T−t is possible, but its drift must be transformed consistently. Do not copy an update derived for decreasing t into an increasing step counter without changing the interpretation. Neural implementations commonly predict noise, clean data, or velocity instead of the score directly. Those predictions require schedule-dependent conversions and input scaling. A tensor with the right shape but the wrong prediction meaning is not a valid drift evaluation, and a scheduler class name cannot repair an incompatible model configuration.

dx=[f(x,t)−g(t)2st(x)] dt+g(t) dWˉt,st=∇xlog⁡pt\mathrm dx=[f(x,t)-g(t)^2s_t(x)]\,\mathrm dt+g(t)\,\mathrm d\bar W_t,\qquad s_t=\nabla_x\log p_t
This reverse formula assumes scalar state-independent g, appropriate regularity and the exact marginal score of the stated forward process. It runs T→0 with dt<0 and backward noise variance |dt|. Coordinates and time are normalized in this lesson; more general diffusion matrices need the corresponding general formula.↔ Scroll wide equations horizontally.

A deterministic flow can transport a random initial law

The probability-flow ODE uses half the score correction in its drift. With the exact score, compatible initial marginal, and regularity conditions, its density evolution matches the forward SDE marginals; integrating it backward provides the corresponding transport. This is a distributional statement, not a pairing that makes each deterministic trajectory equal a reverse-SDE path. Randomness remains in the initial draw even when the subsequent evolution is deterministic. A deterministic map of random inputs need not produce a point mass.

In practice, the learned score has estimation error, terminal noise may only approximate the required p_T, and a finite-step solver adds numerical error. Replacing the exact score with a neural estimate does not automatically preserve the same marginal equivalence between two approximate processes. A DDIM-style deterministic update and a probability-flow interpretation also require their own parameterization and discretization conditions. Do not describe a twenty-step pipeline as exact sampling from the data distribution merely because an ideal continuous-time equation has an exact marginal identity.

dxdt=f(x,t)−12g(t)2st(x)\frac{\mathrm dx}{\mathrm dt}=f(x,t)-\frac12g(t)^2s_t(x)
The probability-flow ODE uses the same forward f, g, and exact p_t score under the stated regularity. It may be integrated backward from the matching terminal distribution. Equality concerns continuous-time marginals, not stochastic paths, learned-score approximations, or arbitrary finite-step solvers.↔ Scroll wide equations horizontally.

Specify which process a serving result actually used

Record the model revision, prediction parameterization, forward schedule, sampler update, terminal initialization, endpoint handling, conditioning, precision, and random draws. Distinguish numerical integration error from model-score and initial-law error when diagnosing artifacts. A smaller step may reduce one discretization component while leaving a poor learned field or incompatible parameterization untouched. Use controlled low-dimensional tests for drift signs and noise variances before trusting an image-level quality score to diagnose a mathematical mismatch.

The probability instrument contrasts a nondegenerate distribution with a degenerate one without pretending to integrate diffusion. Reset, set both p=0.5 and n=10, predict mean five and variance 2.5, then Play or Step. Separately Reset, set both p=1 and n=10, predict mean ten and variance zero, then Play or Step. These exact binomial bars are not Brownian paths, reverse-SDE samples, or ODE transport. They support the limited lesson that a mean alone does not characterize a stochastic law and that deterministic evolution must not be confused with absence of initial randomness.

Check a frozen-coefficient reverse increment

At one normalized scalar state x=1, take f=0, g=2, score s=−0.25, and decreasing-time step Δt=−0.1. Compare one frozen-coefficient Euler–Maruyama reverse increment with the probability-flow Euler drift increment.

  1. The reverse-SDE drift is 0−4×(−0.25)=1. Its drift-only increment is 1×(−0.1)=−0.1, so the one-step conditional mean under frozen coefficients is 0.9.

  2. The reverse stochastic increment has variance g²|Δt|=4×0.1=0.4 and standard deviation √0.4≈0.63246. It must not be assigned negative variance because Δt is negative.

  3. The probability-flow drift is 0−(4/2)×(−0.25)=0.5, giving increment −0.05 and Euler value 0.95. These local approximations neither have identical paths nor prove an exact finite-step marginal match.

The result

Time direction, the factor of one half, and noise variance are independent checks. A correct local calculation is necessary but not a guarantee of exact generated distributions.

Explore the model

Interactive lab

Interactive lab · probability

A stochastic law contains more than its mean

Reset, set both p=0.5 and n=10, predict mean five and variance 2.5, then Play or Step. Separately Reset, set both p=1 and n=10, predict mean ten and variance zero, then Play or Step. The exact binomial model performs no SDE integration, learned scoring, or probability-flow transport.

A stochastic law contains more than its meanThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forDistinguish distributional assumptions, path dynamics, and finite-step approximations.

Check your understanding

What does ideal probability-flow equivalence establish under exact-score and regularity assumptions?

Try it yourself

For f=0, g=1, score −0.4, and Δt=−0.05, compute the reverse-SDE drift increment and the probability-flow drift increment under the same frozen-coefficient convention.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use the full score correction for the reverse SDE and half for the ODE.

Compare with the solution

The reverse drift is 0.4, so its drift increment is −0.02. The probability-flow drift is 0.2, giving −0.01. The SDE additionally has a stochastic increment with variance 0.05; neither drift-only value describes a full random sample or establishes finite-step exactness.

Take it with you

  • Keep reverse-time sign conventions and prediction conversions explicit.
  • Ideal probability-flow equivalence concerns marginals under exact-score assumptions.
  • Finite-step learned-model serving remains approximate and needs empirical validation.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.