Keep generated text and structured proposals outside interpreter authority
Validate output representations, authorize proposed operations, and use the correct boundary for HTML, SQL, shell arguments and other downstream consumers.
On this page
The question for today
What you’ll understand
- Treat model output as untrusted data even when it matches a schema.
- Separate context-specific interpreter handling from resource authorization.
- Bound structured output size and verify the actual downstream effect.
A model’s output can be fluent, valid JSON, and still unsafe to pass into a privileged interpreter. The application chooses whether that string becomes HTML, a database query, a command argument, a file path or a tool operation. Those transitions are enforcement boundaries outside the language model. Validate the expected representation, preserve data/code separation in the chosen API, and authorize the actual action and target. A model confidence score or instruction to behave carefully cannot replace these controls.
Separate well-formed data from permitted operations
Use a bounded schema for a structured proposal, including allowed operation names, field types and collection sizes. Reject unsupported fields and nonfinite numeric values under the actual parser/validator contract. Then resolve resources and check current permissions using trusted caller context. A valid integer document ID may refer to another tenant’s data; a well-formed URL may point to an unapproved destination. Syntactic validity is necessary for the selected interface but does not establish ownership, intent or allowed side effects.
Bound the complete output and each nested collection before expanding it into tasks. A model that emits a thousand small actions can exceed a work budget even if each action is individually valid. Keep a server-side maximum on operation count and cumulative cost, and require any task-specific approval at the actual execution boundary. These are proposed application controls; a structured-output API or JSON Schema implementation does not automatically implement the resource, destination or authorization policy.
Use the consumer’s data interface and its exact context rules
For ordinary UI text, use a text-rendering path. If HTML is intentionally supported, use an appropriate sanitizer and context-specific handling rather than treating arbitrary model HTML as trusted. For SQL, parameterized value binding separates values from query structure, while dynamic identifiers or operation choices need an explicit allowed mapping. A read-only account can reduce authority but does not authorize reading every tenant’s rows. The database must still enforce the actual access policy.
For operating-system work, prefer a narrow library or fixed executable interface with validated arguments over building a shell command string. Avoiding a shell does not eliminate argument injection: a tool can interpret an untrusted argument as an option or another control value. Bound and validate each argument under the chosen program’s contract. Paths, URLs, templates and Markdown have their own interpretation rules, so one generic escaping function cannot establish safety across all downstream contexts.
Use harmless canaries to observe the actual consumer
Test known-safe data containing punctuation, markup-like text, option-like strings and unauthorized target IDs against the exact downstream interface in an isolated fixture. Verify the expected rendered text, bound query values, argument vector or denied operation, not just the model’s answer. Use a stubbed model output when testing deterministic enforcement so a model’s refusal cannot hide a missing application check. Separately measure model behavior on representative prompts without interpreting that behavior as proof of the consumer boundary.
The network lab is a single sigmoid scorer, not an authorization engine. Reset and set both x=3,w=3, predict σ(8.5)≈0.999797, then Play or Step. Reset and set both x=−3,w=3 for σ(−9.5)≈0.00007485. The fixed bias is −0.5. A high score does not grant a principal permission to execute a proposed operation. The instrument trains no model and implements no JSON parser, HTML/SQL/shell interpreter, credential or policy decision.
Work it through
Enforce an encoded output budget
A fictional output field contains exactly 4,000 Unicode euro-sign scalars, each encoded as three UTF-8 bytes. The field’s encoded-byte policy is 8,192 bytes, exclusive of JSON overhead. Its schema separately permits up to 4,096 scalars. Evaluate both stated checks.
The scalar count is 4,000, so this field passes the 4,096-scalar length condition under the specified counting convention.
Its encoded payload is 4000×3=12,000 bytes. That exceeds the independent 8,192-byte field cap by 3,808 bytes; passing the scalar condition does not override it.
Reject or apply the declared bounded-output policy before the field enters its downstream consumer. Even an output passing both size checks would still require context-specific interpretation and any action/resource authorization.
The output passes one length contract and fails another. Neither size check makes generated text or a structured action trusted interpreter input.
The living notebook
Explore the model
Interactive lab · network
Keep a toy model score separate from authority
Reset → set both x=3,w=3 predicts σ(8.5)≈0.999797; Reset → set both x=−3,w=3 predicts σ(−9.5)≈0.00007485. Play/Step displays one fixed-bias sigmoid. It is not a trained authorization model, output validator, downstream interpreter or permission check.
- Product · wx
- 1.2
- Logit · wx − 0.5
- 0.7
- Sigmoid output
- 0.6682
One inspectable sigmoid unit, σ(wx − 0.5). Playback traces the computation graph. There is no training, attention mechanism or hidden deep network. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA confident numerical output cannot confer authority to a generated operation.
Pause & reason
Check your understanding
Your turn
Try it yourself
A field has 3,000 valid Unicode scalars requiring four UTF-8 bytes each, while its encoded-byte cap is 10,000. What is the payload and overage? Would merely escaping it for HTML make it acceptable to a SQL or shell consumer?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Count bytes before any additional representation overhead, then keep consumer contexts distinct.
Compare with the solution
The payload is 3000×4=12,000 bytes, exceeding the cap by 2,000. HTML escaping neither fixes that byte-budget failure nor creates a general SQL or shell data boundary. Each consumer needs its own correct API and validation, with authorization for any action and resource. Extra escaping can also change encoded length, so its representation needs separate bounded handling.
Keep the idea
Take it with you
- Well-formed model output remains untrusted at the next interpreter boundary.
- Use consumer-specific data interfaces and independent operation/resource authorization.
- Measure output units and effects directly instead of trusting model scores or refusals.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.