Preprocessing is part of the learned model and the evidence coordinate system
Version decoding, geometry, scaling, normalization, tokenization, and cache identity so the same media retains a reproducible model meaning.
On this page
The question for today
What you’ll understand
- Calculate a declared rescale-then-normalize transform and diagnose a range mismatch.
- Map a point through a simple resize-and-crop coordinate transform.
- Specify processor and cache versions alongside model and encoder artifacts.
A checkpoint does not consume an abstract image or sound. It consumes arrays produced by decoding, orientation handling, resizing, cropping, rescaling, normalization, and other modality-specific transforms. Changing any of these can change the task evidence or the input distribution while leaving tensor shapes valid. The same concern applies to audio sample rate and text tokenization. Reproducibility and safe cache reuse require this transformation chain to be a versioned part of the model contract, with geometry retained for grounding evaluation.
Keep value ranges and channel conventions explicit
Specify raw value type, channel order, rescale factor, mean, and positive standard deviation for each channel. Applying a normalization designed for values in zero through one directly to byte-scale values creates a very different input. Rescaling twice can also collapse the range. A floating-point array's dtype alone does not reveal whether it already lies on the expected scale. Record and test the actual transform sequence. Versioned Transformers 4.57.1 image-processor documentation exposes rescale and normalization controls, but defaults and supported operations still depend on the selected processor class and checkpoint.
Decode conventions matter too. Orientation metadata, alpha handling, color conversion, interpolation, and antialiasing can alter content or coordinates. Training augmentation and deterministic serving transforms need not be identical, but their relationship must be intentional and evaluated. For audio, sample rate, channel mixing, amplitude convention, segmentation, and feature extraction define another chain. Do not infer compatibility merely because both outputs are floating-point tensors. Statistics fitted from data must respect the intended training and evaluation split rather than using future or held-out information to construct an apparently better preprocessing rule.
Geometry transforms must travel with spatial labels
A resize followed by a crop changes where an object or text region lies in processed coordinates. Transform bounding boxes, points, and masks through the same geometry used for the input. Keep horizontal and vertical scales separate when resizing is anisotropic, and account for padding or crop offsets with the correct sign. A coordinate inverse can recover the original location of retained content under the stated transformation, but cannot reconstruct pixels that were cropped away or disambiguate content lost in downsampling.
Keep original dimensions, selected crop, interpolation version, and the resulting spatial mapping with a grounding example. Test known points near boundaries and check clipping conventions rather than only image centers. A model blamed for missing a small label may never have received a readable version after preprocessing. A benchmark's reference coordinates can also be wrong if they refer to the original image while evaluation interprets them in the resized crop. Separate geometric pipeline mistakes from representation and reasoning limitations before drawing research conclusions.
Cache and publish the complete compatible transformation
An encoder-feature cache key should distinguish immutable media identity, decoder and preprocessing configuration, feature-selection rule, encoder revision, and any representation-changing dtype choice. Otherwise a valid old tensor can be reused under a new model meaning. Save processor artifacts, dependency versions, and numerical fixtures with the checkpoint rollout. Compare expected shapes, selected values, and task outputs after upgrades; a byte checksum can detect a change but cannot tell whether the new behavior is scientifically or operationally correct. Test cohorts sensitive to the changed transform.
The cache lab illustrates eviction only, not preprocessing compatibility. Reset, set both C=4 and stride s=4, predict initial four misses and twelve hits at sixteen requests, then inspect or Play. Reset and set both C=2 and s=4 for sixteen initial misses. The model starts empty, uses fully associative LRU on address(i×s) mod 16, adds two requests per playback second, and caps at 4096. Entries have no processor, encoder revision, media hash, or byte width. Its hits demonstrate toy reuse, not evidence that a production feature cache used semantically compatible values.
Work it through
A value transform and a crop transform need different metadata
A toy processor rescales an 8-bit pixel by 1/255, then uses mean 0.5 and standard deviation 0.25. Separately it scales spatial coordinates by 0.5 on each axis, then crops 16 pixels from the left and 8 from the top of the scaled image. Consider raw pixel 128 and an original retained point(100,80).
The rescaled pixel is 128/255≈0.5019607843. Its normalized value is(128/255−0.5)/0.25≈0.0078431373. Omitting rescaling would instead give(128−0.5)/0.25=510, a different input range.
The point first scales to(50,40), then subtracting crop offsets produces(34,32). Normalizing pixel values did not perform or describe this coordinate transformation.
The inverse for this retained point gives((34+16)/0.5,(32+8)/0.5)=(100,80). This recovers its coordinate under the defined mapping, not any image content discarded outside the crop.
Range, channel, and geometry metadata belong together but describe distinct operations. Correct shapes cannot substitute for a checkpoint-compatible value transform and valid target coordinates.
The living notebook
Explore the model
Interactive lab · cache
A cache hit is useful only under the real representation key
Reset, set both C=4 and s=4, predict four misses and twelve hits at initial N=16, then inspect or Play. Reset and re-enter both C=2 and s=4 for sixteen misses. The empty fully associative LRU model uses address(i×s) mod 16 and N=16+floor(2t), capped at 4096. It has no media identity, preprocessing version, encoder, tensor dtype, or byte accounting.
- Hits / requests
- 0 / 16
- Hit rate
- 0%
- Address-stream period
- 16 addresses
A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA real feature cache must validate representation compatibility in addition to finding a stored entry.
Pause & reason
Check your understanding
Your turn
Try it yourself
A declared transform rescales raw pixel 64 by 1/255, then uses mean 0.4 and standard deviation 0.2. Compute the normalized value. If a crop removes the object needed by the label, can correct normalization or an inverse coordinate map recover that missing evidence?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Apply scaling before subtracting the mean, and distinguish retained coordinates from discarded pixels.
Compare with the solution
The normalized value is(64/255−0.4)/0.2≈−0.74509804. Correct range handling does not reconstruct an object outside the crop. An inverse map can locate retained coordinates but cannot recreate discarded content; the sample or task label needs an appropriate preprocessing policy or exclusion.
Keep the idea
Take it with you
- Decode, value range, geometry, and modality-specific transforms are part of the checkpoint contract.
- Spatial labels must follow the same geometry, while lost content cannot be recovered by coordinate algebra.
- Version processor artifacts and representation cache keys with encoder and model revisions.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.