Count encoder features and language-model slots before admitting media

For a simple nonoverlapping image grid, processed height and width divided by patch size determine the patch count when both are divisible. Padding, cropping, tiling, global thumbnails, class tokens, and feature-selection rules can change the actual output. Raw upload dimensions therefore do not uniquely determine encoded length. A processor may resize first or produce multiple crops. Record the processed shapes and the encoder feature tensor rather than inferring cost from a file's compressed byte size or a single visible placeholder in a chat window.

Other modalities need analogous explicit accounting. A video's sampled frame count and per-frame features matter separately from its duration; an audio encoder's stride determines how waveform samples become feature steps. The learned representation dimension and sequence length are different quantities. Multiplying them estimates tensor values, while the architecture determines how many become language-model positions. A fixed learned-query connector can reduce feature count, but its encoder still processed the original media and its compressed output is not a lossless representation of every detail.

Architectures make different choices. The original BLIP-2 paper uses a learned-query bottleneck, while LLaVA-style projected visual features and Flamingo's visual resampling with gated cross-attention illustrate different bridges. Their lengths and masks are not interchangeable. A language-context calculation that concatenates visual features applies only when those features occupy that sequence. If media is supplied through separate cross-attention, account for its keys, values, and masking separately instead of pretending its raw feature count is automatically the same number of ordinary text tokens.

For a concatenated-prefix contract, budget text, retained visual positions, formatting tokens, and reserved generation against the allowed sequence length. Some systems allocate generated capacity dynamically, but admission still needs a stated policy. Truncating text without awareness of expanded image positions can remove required markers or break feature alignment. Versioned Transformers 4.57.1 LLaVA documentation explicitly describes processor attributes affecting image-token expansion; use that checkpoint's compatible processor and inspect its output rather than adopting an approximate image-token count as a universal constant.

Test a small image, an extreme aspect ratio, multiple crops, long text, and the maximum supported media count. Compare predicted processed length with the processor's feature shapes and the model's merge requirements. Include special-token and generation settings in the trace. Report quality as well as capacity when reducing resolution or connector output length: a smaller input may remove the text, object, or temporal detail the task needs. The correct budget is an enforceable serving contract backed by these probes, not a promise that every admitted request will have equal cost.

The roofline lab offers a separate arithmetic-versus-byte ceiling. Reset, set both I=4 FLOP/byte and B=100 decimal GB/s, predict 400 GFLOP/s, then Play. Reset and set both I=16 and B=100 for the fixed 1000 GFLOP/s cap. It has no media encoder, tokenization, visual-feature merge, context limit, or generation cache; playback traces a selected synthetic roof. Use the worked representation counts for token admission, and use measured encoder and language-model execution for latency rather than interpreting this diagram as a multimodal capacity simulator.