Measure quality with explicit rules and denominators

For a score intended to lie between zero and one, decide whether missing values are allowed, how non-finite values are handled, and whether values outside the interval are rejected or quarantined. A row can satisfy the type schema while violating this application rule. In PostgreSQL, an ordinary CHECK passes when its expression is true or unknown, so a range comparison alone does not necessarily reject NULL. Add the appropriate not-null requirement when absence is invalid. Other engines and validators have their own semantics; write representative cases and inspect the actual evaluated result rather than assuming a Boolean expression has identical behavior everywhere.

Record-level predicates do not express every quality obligation. Uniqueness spans a defined key scope, referential integrity relates datasets, and an expected row count depends on which source events should exist. A foreign key can establish a referenced identifier exists without proving that the associated training label is correct. Likewise, a nonempty string constraint does not establish that a text example is useful or permitted for training. Classify each rule by what it proves, who owns its definition, and which operation enforces it. Keep exceptions versioned rather than changing the interpretation of failure after inspecting an inconvenient batch.

COUNT(*) counts rows, whereas COUNT(column) ignores null values in familiar SQL aggregate semantics. Dividing valid non-null scores by the latter measures quality among present values, not among all submitted rows. Both metrics can be useful if labeled correctly. Also retain counts before and after parsing, filtering, deduplication, and quarantine. A perfect profile of the surviving rows can conceal that a large share never reached the profiler. Expose the admission funnel so an apparent improvement caused by dropping difficult examples cannot be mistaken for a repaired source.

When several rules fail on the same row, adding their individual failure counts can overcount rejected rows. Use a union of failed-row identities for total rejection and separate per-rule counts for diagnosis. The intersection matters, particularly when a broken parse creates several downstream missing fields. Define duplicate handling before comparing counts: an example repeated three times contributes differently to row count, distinct-key count, and a weighted training distribution. Quality metrics should make these differences visible instead of compressing every issue into one percentage whose numerator and denominator change between reports.

A profile can reveal value ranges, missingness, categories, and shifts, but it does not automatically enforce the rule on future writes. Sampling reduces cost while potentially missing rare or localized defects; sampling only one partition or only successful parses can also bias the result. Check important slices such as collection source, language, ingestion hour, and label class, and preserve the profile's source snapshot and selection procedure. A training-quality change can reflect a genuine population shift rather than a broken pipeline, so compare data provenance and the intended population before deciding which values should be repaired.

The sampling instrument shows an exact distribution under a known independent Bernoulli model. For each comparison, Reset, set both p=0.1 and n to 20 or 100, predict the sample-mean variance, then inspect. The respective variances are 0.0045 and 0.0009. Playback highlights possible outcomes rather than collecting samples. This does not turn a convenience sample into representative data or guarantee discovery of a defect confined to an unobserved partition. Its normal comparison is an approximation with explicit limits, while the exact binomial distribution still relies on independent observations with a common probability.