research30 min studyLesson 790 in Engineering

Seeded generation reproducibility boundaries

Control request-owned random streams and complete generation manifests without promising identical pixels across arbitrary numerical environments.

On this page

What you’ll understand

  • Distinguish an initial seed from a generator’s current state and draw assignment.
  • Preserve request streams across batching, retries, and cancellation.
  • Define bitwise, numerical, perceptual, and distributional reproducibility claims separately.

A seed is one input to a generation computation, not a complete description of its output. Models, schedules, guidance, random draw ordering, precision, kernels, batching, and decoding also matter. A serving system should preserve enough state to reproduce the promised level of behavior and state that level clearly. Cross-hardware identical pixels are a much stronger claim than using the same initial integer seed.

A random stream has state and an assignment policy

Initial latent generation and stochastic sampler steps can consume random values. Reusing a generator object after one request starts from its advanced state unless it is explicitly reset or reconstructed. Diffusers v0.35.1 recommends passing generator state and explains this consumption behavior. A deterministic ODE-style sampler may use no new stochastic draws after initialization, but its output still depends on the initial latent. A stochastic sampler can require an entire per-step noise stream in addition to that initial state.

A global generator shared across requests can assign different draws when batch order, step count, cancellation, or retries change. Give each request an explicit stream policy, with stable logical identifiers and a declared retry rule. Separate per-request generators or a supported counter-based mapping can reduce scheduling coupling, but neither is implemented merely by attaching a request ID to a log. If a model uses multiple random sources, include every relevant source and the transformation from random state to tensors. Preserve actual initial latents or noise tensors when that is the needed comparison boundary.

Ur,k,b,j=PRNG⁡(stream⁡r,counter⁡(k,b,j))U_{r,k,b,j}=\operatorname{PRNG}(\operatorname{stream}_r,\operatorname{counter}(k,b,j))
This is a conceptual counter-addressed design: r identifies a request stream, k a logical sampler step, b a stochastic branch, and j an element index. It requires a supported generator and collision-safe domain assignment; it is not a claim that every Diffusers backend uses this mapping or that distinct outputs are statistically independent automatically.↔ Scroll wide equations horizontally.

Numerical execution is another part of the manifest

Record model and autoencoder revisions, tokenizer and conditioning artifacts, adapters, scheduler configuration, realized grid, guidance policy, prediction type, precision, initial latent or generator state, and relevant software/hardware settings. Floating-point reduction order, kernel selection, nondeterministic operations, and batch shape can change results even with the same random draws. CPU-generated initialization can help align one source of randomness across devices but does not force the subsequent denoiser and decoder arithmetic to match bitwise.

PyTorch and the pinned Diffusers reproducibility documentation explicitly limit guarantees across releases and platforms. Deterministic-algorithm settings can improve repeatability in a supported fixed environment while changing performance or rejecting unsupported operations. Define the oracle: exact latent or pixel equality, bounded numerical error, perceptual similarity, or a distributional test. Passing a tolerance for one generated image is not proof that two samplers have the same output law, and bitwise equality on one machine is not a universal portability guarantee.

xout=G(M,C,T,U,N)x_{\rm out}=G(\mathcal M,\mathcal C,\mathcal T,\mathcal U,\mathcal N)
M denotes model/decoder artifacts, C conditioning and policy, T the realized sampler grid/state, U assigned random tensors, and N numerical execution choices. This dependency specification has no units and gives necessary manifest categories, not an injective mapping or cross-platform determinism theorem.↔ Scroll wide equations horizontally.

Test scheduling independence and restoration explicitly

Replay a fixed request alone, in different compatible batch orders, after another request consumes random values, and after a neighboring request is canceled. Verify the intended stream assignment and compare outputs using the declared oracle. Test retries separately: the service may promise the same candidate again or intentionally create a new sample, but that policy should not depend on accidental process-global RNG progress. For resumable generation, preserve the latent, solver history, coordinate, and random state at one consistent boundary rather than only the original seed.

The modular instrument is a transparent sequence-position analogy, not a random generator. Reset, set both m=16 and a=5, predict residue fifteen after three additions, then Step three times. Separately Reset, set both m=16 and a=3, predict residue nine after three additions, then Step three times. Both sequence rule and position affect subsequent values. This short exact periodic orbit has no Gaussian sampling, statistical randomness, device kernels, or model outputs. It helps distinguish a fully specified stream position from the fact that two processes once started at zero.

Shared consumption changes which request gets a draw

Use the deterministic toy sequence formed by repeatedly adding five modulo sixteen from zero. The first four values are 5, 10, 15, and 4. Two requests A and B each need two values; compare sequential and interleaved assignment.

  1. With A consuming two values first, A receives (5,10) and B receives (15,4). The shared sequence itself is fully determined by its starting state and rule.

  2. With interleaving A,B,A,B, A receives (5,15) and B receives (10,4). A’s second value changes from ten to fifteen although the global sequence and initial state are unchanged.

  3. This is not a statistical PRNG example. It demonstrates why a common initial seed does not define per-request draw assignment when scheduling changes. Actual generation needs an implemented request-stream policy plus a scoped numerical reproducibility contract.

The result

Control both stream state and ownership. Identical global initialization is weaker than reproducible request-level random inputs, which is itself weaker than universal bitwise output identity.

Explore the model

Interactive lab

Interactive lab · modular

The sequence rule and position determine the next value

Reset, set both m=16 and a=5, predict residue fifteen after three additions, then Step three times. Separately Reset, set both m=16 and a=3, predict residue nine, then Step three times. This deterministic additive orbit is not a PRNG, Gaussian source, statistical independence test, or cross-device generation comparison.

The sequence rule and position determine the next valueAddition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Current readings: Residue r0: 0; Orbit period · m/gcd(m,a): 12; Distinct residues visited: 1 of 12. All parameters are adjustable with the labeled controls below.Repeated addition modulo 12: the current residue is 0, and the orbit period is 12.01234567891011+ 5mod 12An additive orbitn = 0rₙ = 0Period = 12Cycle starts at 0:0 → 5 → 10 → 3 →8 → 1 → 6 → 11 →4 → 9 → 2 → 7 → 0Exact residues · two additions per second
Additive orbitCurrent residue / next edge
Residue r0
0
Orbit period · m/gcd(m,a)
12
Distinct residues visited
1 of 12
t = 0 s

Addition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forRecord request stream assignment and numerical conditions in addition to the seed.

Check your understanding

Two runs use the same initial seed but differ in hardware, batch shape, and stochastic draw consumption. What can they assume?

Try it yourself

A hypothetical sampler draws one four-element initial noise tensor and two later four-element stochastic noise tensors. Count the abstract generated variates, and explain why this does not necessarily equal the number of underlying uniform RNG operations.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Count tensor elements first, then distinguish a requested distribution from its implementation.

Compare with the solution

The sampler requests 4+2×4=12 abstract noise variates. A backend can generate normal values using vectorized methods, cached intermediates, or other transformations, so twelve variates do not specify twelve uniform draws or a portable raw generator-state offset. Preserve the actual supported state or assigned tensors rather than inventing that equivalence.

Take it with you

  • Treat seeds, current generator states, and request draw assignments separately.
  • Preserve the complete sampler and numerical manifest for the promised oracle.
  • Do not equate fixed randomness with universal cross-hardware or distributional exactness.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.