research34 min studyLesson 799 in Engineering

Preprocessing is part of the learned model and the evidence coordinate system

Version decoding, geometry, scaling, normalization, tokenization, and cache identity so the same media retains a reproducible model meaning.

On this page

What you’ll understand

  • Calculate a declared rescale-then-normalize transform and diagnose a range mismatch.
  • Map a point through a simple resize-and-crop coordinate transform.
  • Specify processor and cache versions alongside model and encoder artifacts.

A checkpoint does not consume an abstract image or sound. It consumes arrays produced by decoding, orientation handling, resizing, cropping, rescaling, normalization, and other modality-specific transforms. Changing any of these can change the task evidence or the input distribution while leaving tensor shapes valid. The same concern applies to audio sample rate and text tokenization. Reproducibility and safe cache reuse require this transformation chain to be a versioned part of the model contract, with geometry retained for grounding evaluation.

Keep value ranges and channel conventions explicit

Specify raw value type, channel order, rescale factor, mean, and positive standard deviation for each channel. Applying a normalization designed for values in zero through one directly to byte-scale values creates a very different input. Rescaling twice can also collapse the range. A floating-point array's dtype alone does not reveal whether it already lies on the expected scale. Record and test the actual transform sequence. Versioned Transformers 4.57.1 image-processor documentation exposes rescale and normalization controls, but defaults and supported operations still depend on the selected processor class and checkpoint.

Decode conventions matter too. Orientation metadata, alpha handling, color conversion, interpolation, and antialiasing can alter content or coordinates. Training augmentation and deterministic serving transforms need not be identical, but their relationship must be intentional and evaluated. For audio, sample rate, channel mixing, amplitude convention, segmentation, and feature extraction define another chain. Do not infer compatibility merely because both outputs are floating-point tensors. Statistics fitted from data must respect the intended training and evaluation split rather than using future or held-out information to construct an apparently better preprocessing rule.

zc=axc−μcσc,σc>0z_c=\frac{a x_c-\mu_c}{\sigma_c},\qquad \sigma_c>0
This chosen transform rescales raw channel x by a before normalizing with compatible mean mu and positive standard deviation sigma. All range conventions and operation order must match the checkpoint. It is an illustrative formula, not a universal processor default; clipping, quantization, or other transforms need their own specification.↔ Scroll wide equations horizontally.

Geometry transforms must travel with spatial labels

A resize followed by a crop changes where an object or text region lies in processed coordinates. Transform bounding boxes, points, and masks through the same geometry used for the input. Keep horizontal and vertical scales separate when resizing is anisotropic, and account for padding or crop offsets with the correct sign. A coordinate inverse can recover the original location of retained content under the stated transformation, but cannot reconstruct pixels that were cropped away or disambiguate content lost in downsampling.

Keep original dimensions, selected crop, interpolation version, and the resulting spatial mapping with a grounding example. Test known points near boundaries and check clipping conventions rather than only image centers. A model blamed for missing a small label may never have received a readable version after preprocessing. A benchmark's reference coordinates can also be wrong if they refer to the original image while evaluation interprets them in the resized crop. Separate geometric pipeline mistakes from representation and reasoning limitations before drawing research conclusions.

x′=sxx−cx,y′=syy−cy,x=x′+cxsx,y=y′+cysyx'=s_xx-c_x,\qquad y'=s_yy-c_y,\qquad x=\frac{x'+c_x}{s_x},\quad y=\frac{y'+c_y}{s_y}
The declared transform scales original coordinates by positive s_x,s_y and then crops an offset(c_x,c_y) measured in scaled-image pixels. The inverse applies to retained locations under this affine convention. Padding, rotation, nonlinear warps, clipped regions, and pixel-center conventions require their actual additional mapping.↔ Scroll wide equations horizontally.

Cache and publish the complete compatible transformation

An encoder-feature cache key should distinguish immutable media identity, decoder and preprocessing configuration, feature-selection rule, encoder revision, and any representation-changing dtype choice. Otherwise a valid old tensor can be reused under a new model meaning. Save processor artifacts, dependency versions, and numerical fixtures with the checkpoint rollout. Compare expected shapes, selected values, and task outputs after upgrades; a byte checksum can detect a change but cannot tell whether the new behavior is scientifically or operationally correct. Test cohorts sensitive to the changed transform.

The cache lab illustrates eviction only, not preprocessing compatibility. Reset, set both C=4 and stride s=4, predict initial four misses and twelve hits at sixteen requests, then inspect or Play. Reset and set both C=2 and s=4 for sixteen initial misses. The model starts empty, uses fully associative LRU on address(i×s) mod 16, adds two requests per playback second, and caps at 4096. Entries have no processor, encoder revision, media hash, or byte width. Its hits demonstrate toy reuse, not evidence that a production feature cache used semantically compatible values.

A value transform and a crop transform need different metadata

A toy processor rescales an 8-bit pixel by 1/255, then uses mean 0.5 and standard deviation 0.25. Separately it scales spatial coordinates by 0.5 on each axis, then crops 16 pixels from the left and 8 from the top of the scaled image. Consider raw pixel 128 and an original retained point(100,80).

  1. The rescaled pixel is 128/255≈0.5019607843. Its normalized value is(128/255−0.5)/0.25≈0.0078431373. Omitting rescaling would instead give(128−0.5)/0.25=510, a different input range.

  2. The point first scales to(50,40), then subtracting crop offsets produces(34,32). Normalizing pixel values did not perform or describe this coordinate transformation.

  3. The inverse for this retained point gives((34+16)/0.5,(32+8)/0.5)=(100,80). This recovers its coordinate under the defined mapping, not any image content discarded outside the crop.

The result

Range, channel, and geometry metadata belong together but describe distinct operations. Correct shapes cannot substitute for a checkpoint-compatible value transform and valid target coordinates.

Explore the model

Interactive lab

Interactive lab · cache

A cache hit is useful only under the real representation key

Reset, set both C=4 and s=4, predict four misses and twelve hits at initial N=16, then inspect or Play. Reset and re-enter both C=2 and s=4 for sixteen misses. The empty fully associative LRU model uses address(i×s) mod 16 and N=16+floor(2t), capped at 4096. It has no media identity, preprocessing version, encoder, tensor dtype, or byte accounting.

A cache hit is useful only under the real representation keyA fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Current readings: Hits / requests: 0 / 16; Hit rate: 0%; Address-stream period: 16 addresses. All parameters are adjustable with the labeled controls below.Fully associative LRU cache after 16 deterministic requests: 0 hits and 16 misses.Most recent 16 requestsaddress = i × 3 mod 160M3M6M9M12M15M2M5M8M11M14M1M4M7M10M13MCurrent cache order · oldest → newest471013Latest address 13 · miss: inserted as newestToy address stream · period 16 · starts empty
Cached addressMiss / evictionMost recent request
Hits / requests
0 / 16
Hit rate
0%
Address-stream period
16 addresses
t = 0 s

A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA real feature cache must validate representation compatibility in addition to finding a stored entry.

Check your understanding

An encoder cache returns a tensor with the expected shape after the preprocessing configuration changes. What remains necessary?

Try it yourself

A declared transform rescales raw pixel 64 by 1/255, then uses mean 0.4 and standard deviation 0.2. Compute the normalized value. If a crop removes the object needed by the label, can correct normalization or an inverse coordinate map recover that missing evidence?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Apply scaling before subtracting the mean, and distinguish retained coordinates from discarded pixels.

Compare with the solution

The normalized value is(64/255−0.4)/0.2≈−0.74509804. Correct range handling does not reconstruct an object outside the crop. An inverse map can locate retained coordinates but cannot recreate discarded content; the sample or task label needs an appropriate preprocessing policy or exclusion.

Take it with you

  • Decode, value range, geometry, and modality-specific transforms are part of the checkpoint contract.
  • Spatial labels must follow the same geometry, while lost content cannot be recovered by coordinate algebra.
  • Version processor artifacts and representation cache keys with encoder and model revisions.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.