State which part of the input–outcome distribution changed

Covariate shift means the input distribution changes while the conditional outcome distribution given the recorded input remains the same. Label shift, also called prior-probability shift, instead preserves the input distribution conditional on the class while class prevalence changes. Concept shift here means a change in the conditional outcome distribution given inputs. This last definition is broad: label shift can change the posterior outcome probability too. These descriptions are therefore assumptions about factorizations, not three mutually exclusive bins into which every incident must fit.

The recorded input representation matters. A change that looks like conditional-outcome drift after dropping a relevant variable may look like a changed mixture after including it. Likewise, a changed label definition creates a different target rather than merely harder examples for the old target. Preserve schema, target, population, and measurement contracts when comparing distributions. State whether a claimed conditional equality is a modeling assumption or has been checked on sufficiently mature labeled data. Finite observations can support or challenge that assumption without proving equality everywhere.

Under covariate shift, target risk can be expressed as a weighted source expectation when the target input support is covered by the source. The weight is the target-to-source input density ratio. This equality does not make estimated weights reliable: sparse regions and large ratios increase estimator variability. A target population with input regions absent from the source cannot be evaluated by this identity using source outcomes alone. Weight clipping can stabilize an estimator, but it changes the estimand or introduces bias that must be assessed.

Under label shift, a fixed classifier's class-conditional prediction rates can connect observed predicted-class frequencies to target class prevalence. For a binary classifier, an invertible confusion matrix is required to recover the two prevalences uniquely by this method. A classifier that predicts positive equally often for both true classes supplies no such information. Even an invertible estimate may be unstable when those rates are close. The label-shift assumption must be justified independently; a changed prediction rate alone cannot distinguish a new prior from changed class-conditional inputs or a pipeline defect.

Consider two populations with identical input distributions but opposite binary labels at each input. Every input-only test sees the same population, while a fixed predictor can go from correct to incorrect. Conversely, a large input shift can occur entirely in dimensions irrelevant to a good predictor. Use distribution alerts as leads and connect them to outcome evidence, mechanism knowledge, or a carefully scoped assumption. If labels are delayed, explicitly carry that uncertainty into the response instead of calling a convenient unsupervised statistic a performance estimate.

The probability instrument helps separate a class proportion from an outcome relationship. Reset, set p=0.25 and n=20, predict mean count five, then Play or Step. Separately Reset, set p=0.50 and n=20, predict mean count ten, then Play or Step. Its exact binomial bars illustrate a changed marginal prevalence under independent trials. They contain no inputs, classifier, confusion matrix, or conditional labels, so they cannot establish label shift rather than another mechanism. The missing conditional information is precisely what a production diagnosis must supply.