Validate bounded representations before expensive inference work
Combine transport, decoded media, tensor, tokenizer and generation limits while keeping syntactic validation separate from authorization and model behavior.
On this page
The question for today
What you’ll understand
- Place distinct limits before parsing, expansion, decoding and inference.
- Count actual tokenizer and multimodal work under the selected model configuration.
- Reject invalid ranges and shapes without claiming schema validation grants permission.
An inference request changes representation several times: network bytes become parsed fields, decoded media, token IDs, batches, and accelerator work. A bound at one stage does not automatically constrain the next. A small image can decode into many pixels, and equal character lengths can produce different token counts. Specify limits with their units and enforcement points, including what happens when a request exceeds them. Validation establishes that data meets a contract; it does not decide whether the caller may use a model or tool.
Bound parsing and media expansion before constructing tensors
Apply an ingress byte limit and bounded parsing depth or field counts before building an unbounded object tree. For compressed bodies or uploaded media, enforce decoded-byte, pixel, frame, duration, and elapsed-work budgets appropriate to the supported codecs. A reported width or duration is untrusted until checked against the actual decode path. Count cumulative work across multiple attachments, not only per-item maxima, and reject inconsistent or unsupported representations without echoing sensitive content into error logs.
JSON Schema 2020-12 can express types, numeric ranges, array lengths, and string limits, but it does not replace transport budgets or semantic checks involving multiple fields. Its format annotation vocabulary does not by default require format assertion. Record the selected validator and configuration and test the desired behavior explicitly. A syntactically valid URL, identifier, or JSON operation is still not proof that the destination is allowed or that its requested action is authorized.
Use the actual tokenizer and complete context construction
Tokenize with the approved tokenizer version and bound the resulting token count, including trusted templates, conversation history, retrieved text and tool results. Do not estimate a hard limit solely from character length. For multimodal models, account for the actual representation of images, audio or video under the selected preprocessing policy; a count of text tokens alone can omit substantial work. The service should choose an explicit reject or documented truncation policy rather than silently changing meaning at an arbitrary boundary.
Validate generation parameters as finite values with allowed ranges and enforce a maximum output budget. A request fitting the model context can still exceed a tenant or physical-work budget. Conversely, a rate limit does not make malformed tensor dimensions valid. Verify shape/dtype/device compatibility before an expensive kernel path and check dimension products with overflow-safe arithmetic. Treat limits advertised by a framework as pinned API contracts to test, not as a complete security policy for an exposed endpoint.
Reject with an observable boundary and bounded failure cost
Create fixtures at the maximum and just beyond it for body bytes, nesting, decoded pixels, batch dimensions, prompt tokens and output reservations. Include contradictory declared media sizes and nonfinite numeric parameters. Observe where rejection occurs and what CPU, memory, decoder and device work already happened. A request denied after allocating a giant image has not demonstrated the intended preallocation protection. Errors should remain bounded, and temporary buffers and reservations should follow a verified cleanup path.
The queue lab isolates the consequence of accepted work arriving faster than it is serviced. Reset, set both λ=3,μ=2, predict backlog eight at t=4 from initial four, then take eight Steps. Reset and set both λ=2,μ=3 predicts zero at t=4. Edits retain elapsed time and recompute from the initial four, so these are separate runs. The fluid model has no body parser, tokenizer, media decoder, finite admission cap or percentile latency; use actual boundary tests for those controls.
Work it through
Apply independent pixel and context budgets
A fictional endpoint decodes an 8,000×8,000 RGB image into one-byte channels. Its per-image pixel-buffer policy is 128 MiB. Separately, a text request has 3,500 counted prompt tokens, reserves 700 output tokens, and uses a 4,096-token context. Assume the counts are complete under this fixture.
The pixel buffer requires 8000×8000×3×1=192,000,000 bytes, approximately 183.1055 MiB. It exceeds the 128-MiB pixel-buffer cap even if the compressed file was small.
The text reservation is 3500+700=4200 tokens, exceeding the context limit by 104. A body-byte pass does not override this independent token check.
Reject each offending request at its corresponding enforced boundary, before the expensive work that boundary is designed to prevent. Do not claim that these two checks alone bound every decoder allocation or the full inference peak.
Both fictional requests fail different representation budgets. A complete endpoint contract states each unit, enforcement point, and remaining resource scope.
The living notebook
Explore the model
Interactive lab · queue
Compare fresh accepted-work rates with a limited fluid model
Reset → set both λ=3,μ=2; predict q(4)=8 and take eight half-second Steps. Reset → set both λ=2,μ=3 predicts q(4)=0. Edits retain time, so reset for each rate pair. This initial-backlog-four fluid model has no parser, decoded-media cap, tokenizer, admission rejection or p99 measurement.
- Fluid backlog
- 4
- Net rate · λ − μ
- −1 /s
- M/M/1 mean residence
- 1 s
Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forBounding individual representations and bounding accepted work are separate necessary design tasks.
Pause & reason
Check your understanding
Your turn
Try it yourself
A context limit is 4,096 tokens. The complete constructed prompt counts 3,000 tokens, and the requested output reservation is 1,200. What is the overage, and what is the largest output reservation satisfying this context condition alone?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use actual token counts, then subtract prompt size from context capacity. Other limits may still be stricter.
Compare with the solution
The combined count is 4,200, an overage of 104 tokens. The largest context-fitting output reservation is 4096−3000=1,096 tokens. The service can reject or apply a documented permitted adjustment, but should not silently pretend the 1,200 reservation fits. Tenant quotas, physical capacity, generation parameter validation and authorization remain separate checks.
Keep the idea
Take it with you
- Bound bytes, parsed structures, decoded media, tensors and tokens at their own stages.
- Count the complete prompt with the selected tokenizer and multimodal convention.
- Validation, resource admission and authorization have separate oracles.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.