Resolution and packing change token geometry, masking, and actual kernel work

Choose whether images are resized, cropped, or padded to patch-compatible dimensions. Under padding to whole patches, the grid uses the ceiling of height over patch size times the ceiling of width over patch size. A crop or resize has a different geometry. Preserve original-to-processed coordinates for grounding targets and report aspect-ratio cohorts. High resolution can recover small text or details, but it also changes sequence length and position behavior. Increasing raw upload pixels without checking the processor may have no effect if it always resizes to the same grid.

NaViT provides an original example of processing different aspect ratios and resolutions with patch packing and appropriate attention/position treatment. This is an architecture and training design, not permission to send arbitrary sizes into every fixed-grid vision checkpoint. A model may require learned positional interpolation, a restricted crop scheme, or particular resolution buckets. Extra class or summary features, thumbnails, and multiple crops add tokens beyond the bare grid. Match these choices with the checkpoint and its training regime before attributing a quality or efficiency change to resolution alone.

Packing several images into one sequence can reduce unused token slots, but it must preserve each image's identity and positions. A block-diagonal attention mask can prevent one image from reading another when independent image encoding is intended. Class or pooling tokens must also attend only to their intended content. A mask bug can improve a training objective through cross-example leakage while invalidating the independent-image interpretation. Test different packed neighbors and verify that an image's output remains appropriately unchanged under the declared isolation contract.

Logical attention pairs within separate blocks total the sum of squared image lengths. A dense implementation that forms the whole packed score matrix may still allocate or compute the square of the summed lengths, even if it masks cross-image entries afterward. Block-aware or segmented kernels can change that work, but support and achieved efficiency are implementation-specific. Likewise, padding to the maximum length can perform masked work unless the kernel skips it. Report the actual execution method, peak memory, and measured completed time rather than equating logical pair counts with latency.

Compare resolution and packing settings on the same task and input cohort. Keep source content, crop rules, output labels, and evaluation ownership consistent. Inspect small objects, OCR, aspect extremes, and crowded scenes rather than only an aggregate caption score. Measure preprocessing, encoder execution, connector length, and downstream language costs separately. A packing configuration can improve accelerator utilization while making a few oversized images dominate a batch's waiting time. Include admission and batching policies when turning an encoder result into a serving throughput claim.

The roofline lab illustrates one ceiling change without implementing packing. Reset, set both I=4 FLOP/byte and B=50 decimal GB/s, predict 200 GFLOP/s, then Play. Reset and set both I=8 and B=50 for 400. The fixed compute cap is 1000 GFLOP/s and playback traces the synthetic min(1000,B×I) roof. There are no image grids, positional encodings, block masks, padding, or variable-length kernels. Use the explicit length and score-position calculations to form a hypothesis, then validate whether the actual implementation removes the presumed work.