Security evaluation and response

Start from a threat model and an explicit claim, such as a tenant cannot retrieve another tenant's documents through any exposed retrieval path. Define authorized scope, synthetic assets, isolated test environments, rate limits, and stop conditions. Use inert tools where a real external effect is unnecessary. The assessment should not create new privacy leaks or operational incidents simply to prove that a failure is conceivable.

Combine deterministic boundary tests with adversarial and exploratory cases. Include legitimate tasks so a defense that refuses everything does not appear successful. Preserve the system version, model identity, dataset and prompt variants, tool policy, seeds where meaningful, and expected outcomes. A failing trace should identify the component that crossed the boundary, making it possible to write a focused regression test after remediation.

Zero failures in a finite suite do not imply zero risk. For independent identical cases, a one-sided exact upper confidence bound with no failures solves (1 − p) to the power n equals alpha. This frequentist construction describes coverage over repeated experiments, not a posterior probability that the true rate lies below the number. More importantly, its target is the suite's sampling distribution, which may omit important attack families.

Report both statistical uncertainty and coverage gaps. Correlated paraphrases of one case may provide far less diversity than their count suggests. Separate automated judgments from human-reviewed consequences and assess whether the judge misses subtle unauthorized effects. Rerun relevant cases after model, schema, retrieval, permission, or serialization changes. A test result should remain attached to the exact configuration it supports rather than becoming a permanent product-wide safety badge.

Map response actions to each asset: revoke a workload credential, disable a specific tool, quarantine a model release, stop a dataset import, or temporarily serve a restricted capability. These actions have different consequences and should not depend on improvising broad administrator access during a crisis. Keep an owner, a communication path, and tested runbooks. Preserve minimal authorized evidence while avoiding wholesale copying of sensitive prompts and documents.

Containment limits ongoing effects; eradication removes the cause; recovery restores an accepted state and validates it. An old model rollback is insufficient if compromised data will immediately retrain a new candidate or stolen credentials still work. Use post-incident review to update the threat model, regression suite, release gates, and monitoring. Track corrective work through completion and verify that the original failure is closed without silently breaking legitimate workflows.