Roll out a contract without hiding rejected or unchecked data

A contract should identify its dataset and field scope, semantic version, rule, owner, evaluation stage, and failure action. Decide whether a violation rejects a write, quarantines a record or partition, blocks publication, or raises a nonblocking alert. Different actions protect different consumers and create different operational costs. State how existing data is assessed: a newly checked producer does not retroactively validate an archive. Schema compatibility and content quality also need separate gates. A structurally compatible migration can still change label meaning, while a new quality requirement can reject otherwise decodable historical examples.

Database mechanisms illustrate the distinction between future enforcement and historical validation. For supported PostgreSQL CHECK or foreign-key constraints, NOT VALID can defer checking the pre-existing rows while enforcing the new constraint on new changes; VALIDATE CONSTRAINT later addresses the old population according to that mechanism's contract. This is not a universal behavior of every validator, constraint type, or storage engine. A lakehouse rule documented in a catalog may require a separate scan and a producer or publication gate. Verify the actual enforcement point and keep historical-unverified state explicit until the defined validation completes.

Begin with a shadow assessment when appropriate: evaluate the proposed rule and record would-fail outcomes without pretending the existing publication now enforces it. Review representative failures, including rare slices and intentional legacy cases, and compare downstream training behavior under the proposed exclusion. A rule with many false positives can remove valuable or difficult examples, changing the learning population while improving a superficial pass-rate metric. Ground-truth review has its own scope and uncertainty; one convenient sample does not certify every flagged partition. Record the evidence supporting the threshold and exceptions.

A canary then exercises the real gate, quarantine path, error reporting, and affected consumers on a deliberately limited scope. Test both invalid rejection and valid acceptance. Quarantine requires an owner, retention, access policy, review capacity, and replay procedure; otherwise it can become an untracked second dataset or an ever-growing queue. Version the rule and produced artifact so a consumer can distinguish releases, and retain the admission funnel. Expanding enforcement should be a concrete decision based on observed outcomes, not an automatic reward for seeing no errors among rows that were allowed through.

Rollback must consider the data already produced, not just restore an old code binary. New writers may have emitted schemas an old reader cannot consume; quarantine may contain records needing repair rather than blind replay; a reverted rule can reopen a use boundary it was meant to enforce. Define which versions consumers may read, how corrections are published, and which evidence permits republishing a held artifact. Re-run checks against pinned inputs and current required policy before releasing repairs. Never rewrite a failed record silently while leaving its original dataset identity and provenance unchanged.

The queue lab isolates quarantine review capacity. Compare separate fixed-rate runs at arrival 4/service 2 and arrival 2/service 4, interpreting units as review tasks per second. Before each run, Reset, re-set both rates, predict from four initial tasks, then Play or Step. The second fresh backlog drains in two seconds; changing controls alone does not switch the old backlog's rates in place. This fluid model has no validator, ground truth, or publication gate. It can reveal a capacity imbalance, while the contract's actual correctness still depends on rule semantics, outcome tracking, and verified consumer behavior.