Validate enforcement independently of model cooperation

For authorization, create an allowed principal/resource pair and a denied pair while holding the operation otherwise valid. For resource limits, test a supported representation at the boundary and one just beyond it. For loader policy, pair approved benign tensors with a prohibited format or code path without executing unknown code. A denied result is meaningful only when a matched positive case confirms the route and dependency can perform the intended harmless action. This rules out broad service failure as the entire explanation.

Stub the model output when testing deterministic permission and interpreter controls. An intentionally proposed forbidden operation should reach the application’s policy boundary and be rejected there, even if the real model normally refuses to propose it. Inspect an isolated backend canary or ledger to confirm no prohibited effect occurred. Keep separate tests for actual model behavior, since those measure a different, stochastic component and can change with prompts, decoding settings or model versions.

Run the same controlled checks through direct, streaming, batch, fallback and result-retrieval paths as applicable. Include warm caches, already loaded adapters, revoked permissions, wrong token audience, credential expiry and a policy change while work is queued. Observe cancellation and reservation cleanup at the worker, not only at the HTTP layer. A test harness must preserve authenticated context and exact resource identity so it cannot accidentally test an easier or different path than the production one.

Introduce one bounded fault at a time in an isolated environment, such as an unavailable policy store or audit collector, and verify the declared fail-closed, lease or buffering behavior. A test that observes one control rejecting an operation does not prove another control would reject it independently. Where practical, disable or bypass one test-only guard to show its oracle detects the intended failure, then restore the reviewed configuration. Treat shared dependencies and common configuration mistakes as potential correlated failures, not independent protective probabilities.

An acceptance record should identify model/code/library versions, artifact manifests, policy configuration, identities, fixture corpus, expected outcomes, backend observations and any unresolved scope limits. State which changes invalidate the evidence: a new fallback, adapter feature, parser version, cache layer or credential contract can create another boundary. NIST control-assessment methods support explicit objectives and scoped examination/testing; they do not turn a checklist score into a guarantee for all future configurations or attacks.

The probability lab illustrates only the stipulated independent-trial arithmetic. Reset, set both p=0.1,n=10, predict one expected failure and zero-failure probability 0.9¹⁰≈0.34868, then Play or Step. Reset and set both p=0.1,n=20 predicts two expected failures and 0.9²⁰≈0.12158. Its exact binomial bars are not random test executions. The instrument contains no serving endpoint, adversary, fixture oracle or measured failure distribution; do not assign its chosen p to a real defense without justified evidence.