Feature lineage and model-consumer audits

Connect source dataset identity and snapshot, relevant fields, transformation code and artifacts, feature definition and materialization generation, training dataset release, model artifact, and serving deployment. Distinguish structural lineage from one execution’s observed inputs and outputs. Dynamic feature selection, request-time transformations, side inputs, and fallback paths can introduce dependencies absent from a static registry. Use stable namespace-qualified identifiers so similarly named tables, feature views, and model versions do not collapse into one misleading node.

OpenLineage 1.53 models datasets, jobs, and runs with explicit identities and extensible facets. Feast exposes an OpenLineage integration for supported operations, but an integration does not prove every consumer or external transformation is instrumented. Add application-level relationships to model artifacts and request resolution where necessary, and validate that events arrive with the expected completeness. A green job-run event describes an operation’s reported state, not the truth of every upstream value or a guarantee that no undocumented model read it.

A source defect may affect only a partition, entity class, time interval, or transformation revision. Intersect that scope with materialization runs, model-training snapshots, and request-time feature resolutions. A model can be reachable but unaffected in a particular incident because it used an earlier clean generation or made no requests during the bad interval. Conversely, a model absent from an incomplete catalog can still have consumed the data. Preserve the distinction between confirmed exposure, potential exposure, and unobserved consumers rather than using an empty trace result as proof of nonuse.

Training impact and serving impact also differ. A corrected online feature does not remove its influence from a model trained on a contaminated dataset. A model rollback does not necessarily fix ongoing bad feature serving for other consumers. An audit should name the required actions separately: feature correction, affected dataset rebuild, reevaluation, retraining where justified, and deployment changes. Retain immutable release and run identifiers so a later investigation can reconstruct the evidence even after active registry definitions or model routes change.

Choose a synthetic bad source revision and follow it through a test materialization, dataset, model, and request. Check whether the audit finds all expected consumers, deduplicates models reachable by multiple paths, respects time filters, and identifies intentionally omitted instrumentation. Keep access controls and sensitive identifier handling appropriate for the recorded metadata. Validate lineage edges against actual code and usage periodically; old declarations can remain after a consumer is retired, while a new direct database read can bypass the registry altogether.

The probability instrument illustrates why sampled requests can miss exposure. Reset, set both p=0.2 and n=10, predict two affected requests on average and zero-affected probability 0.8¹⁰, then Play or Step. Separately Reset, set both p=0.2 and n=20, predict mean four and zero-affected probability 0.8²⁰, then Play or Step. These exact iid binomial probabilities assume a known common exposure chance and independent sampling. The display neither discovers lineage nor samples actual requests, and correlated traffic or unknown affected fractions invalidate that simple coverage interpretation.