Conditioning paths: identify where time and content enter the denoiser
Trace timestep embeddings, cross-attention, spatial inputs, and adaptive normalization through architecture-specific diffusion backbones.
On this page
The question for today
What you’ll understand
- Distinguish time conditioning from external semantic conditioning.
- Derive a scoped cross-attention or adaptive-normalization calculation.
- Test shape, mask, and gradient-path correctness without assuming every backbone shares one design.
A denoiser must know the noise level and, for a conditional task, receive the intended content information. U-Net and transformer backbones can inject these signals through different paths. Write the tensor shapes and transformations at each entry point. A conditioning argument accepted by an API does not prove it reaches the network correctly or carries the semantics expected by a checkpoint.
Time and content are distinct signals with explicit encodings
A timestep input may be an integer schedule index, a continuous noise coordinate, or a transformed quantity such as log-SNR. Its embedding must match the model's training convention. Two schedules with the same integer index can represent different corruption levels. External conditions can include class labels, text tokens, spatial masks, or other structured inputs, each with its own encoder and alignment. Do not substitute a scalar class ID for a text embedding or change time units while preserving only the tensor shape.
A spatial condition can be concatenated to image or latent channels, while token sequences can enter through cross-attention. Some architectures combine several mechanisms. Preserve batch correspondence, image-coordinate transforms, padding masks, and null representations after augmentation or cached preprocessing. Freezing an encoder's parameters does not automatically fix its dropout mode or tokenizer revision. A model can silently ignore a malformed condition and still reduce denoising loss, so inspect intermediate shapes and controlled condition changes rather than relying only on successful execution.
Adaptive normalization can modulate and gate residual work
A condition embedding can produce featurewise scale and shift values for normalized hidden features. A separate coefficient can gate a residual branch. DiT studies conditioning choices including adaptive layer normalization and an initialization variant that starts selected modulation or residual outputs at zero. Those are concrete architecture and initialization decisions, not a claim that every diffusion transformer uses the same block. A learned scale or residual gate need not be a probability or lie between zero and one.
Cross-attention and adaptive normalization also expose different capacity and cost tradeoffs. Text length changes attention work, whereas a pooled condition vector can discard token-level distinctions. Spatial concatenation preserves another alignment structure. Compare these paths with matched architecture and training budgets where possible, and report their actual conditioning encoders. A larger condition representation can improve one task while increasing memory, introducing leakage from an unintended input, or making the null branch inconsistent with inference.
A controlled probe should reveal both usage and correctness
Hold noisy input and time fixed while changing only the condition, then inspect outputs and intermediate activations. Shuffle conditions as a diagnostic, compare real and null paths, and verify gradients reach the intended trainable modules. A response difference establishes sensitivity, not correct semantic alignment or improved sample quality. Check masking and spatial transforms on hand-labeled examples. For checkpoint interchange, verify encoder outputs, normalization, patch/grid geometry, time convention, and target parameterization together rather than matching only parameter names.
The network lab shows one bounded scalar gate. Reset, set x=1 and w=1, predict σ(0.5)≈0.622459, then Play or Step. Separately Reset, set x=1 and w=2, predict σ(1.5)≈0.817574, then Play or Step. It has no timestep embedding, attention, normalization, or training. Its sigmoid range should not be transferred to the unconstrained coefficients in the teaching residual block. Use the instrument only to inspect one explicit signal path before deriving the actual backbone's transformations.
Work it through
Trace an explicit modulated residual
For a teaching block, take normalized features [−1,1] and residual input [2,4]. Use gamma=[0.5,0.5], beta=[0.1,0.1], alpha=[0.25,0.25], and F equal to the identity. Treat the given normalized features as exact inputs to this calculation.
Featurewise modulation gives h'=[1.5×(−1)+0.1,1.5×1+0.1]=[−1.4,1.6].
The gated branch is 0.25×[−1.4,1.6]=[−0.35,0.4].
Adding the residual [2,4] gives output [1.65,4.4]. A different gate or normalization definition changes this operation; these numbers are not a measurement of a trained DiT.
The condition can change scale, shift, and residual contribution through separate paths whose dimensions and semantics must all match.
The living notebook
Explore the model
Interactive lab · network
Inspect one bounded signal path
Reset, set x=1 and w=1, predict output about 0.622459, then Play or Step. Separately Reset, set x=1 and w=2, predict about 0.817574, then Play or Step. The fixed sigmoid has no attention, time embedding, adaptive normalization, or trainable diffusion backbone.
- Product · wx
- 1.2
- Logit · wx − 0.5
- 0.7
- Sigmoid output
- 0.6682
One inspectable sigmoid unit, σ(wx − 0.5). Playback traces the computation graph. There is no training, attention mechanism or hidden deep network. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA conditioning path needs an explicit transformation; sensitivity alone does not establish semantic correctness.
Pause & reason
Check your understanding
Your turn
Try it yourself
One cross-attention query has two already-scaled logits [0,ln(3)] and scalar values [2,6], with neither position masked. Compute its softmax weights and output.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Exponentiating gives unnormalized weights one and three.
Compare with the solution
Weights are [1/4,3/4], so output is 0.25×2+0.75×6=5. This hand computation assumes the scaled logits were already formed correctly and does not validate projections, padding, or the rest of a diffusion block.
Keep the idea
Take it with you
- Time encoding and external content encoding have separate contracts.
- Cross-attention and adaptive normalization are architecture-specific mechanisms.
- Validate shapes, masks, alignment, gradients, and semantics independently.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.