advanced30 min studyLesson 740 in Mathematics

Sensitivity analysis and partial identification

Report what observed data determine under bounded outcomes and show how explicit extra assumptions narrow the answer.

On this page

What you’ll understand

  • Derive worst-case bounds for a binary-outcome ATE without exchangeability.
  • Distinguish an identified set from a confidence interval.
  • Translate a specified untreated-selection bias into an ATT sensitivity range.

A causal question can remain partly answerable when point-identifying assumptions are not credible. Partial identification describes the set of effects compatible with the observed distribution and stated restrictions. Sensitivity analysis instead varies particular untestable assumptions and records how conclusions change. Both approaches make the missing information explicit. A narrow sampling interval around a confounded contrast does not replace either task, because increasing sample size does not reveal counterfactual outcomes that the observed law never identifies.

Bound the missing potential-outcome means

Consider binary treatment and potential outcomes constrained to the interval zero through one. Assume consistency and no interference, with both observed treatment groups having positive probability, but do not assume exchangeability. Let p be the treatment prevalence and μ₁,μ₀ the observed arm means. The treated potential-outcome mean averages its known treated-group component and its unknown untreated-group component. Bound that unknown component between zero and one. Do the corresponding calculation for the untreated potential-outcome mean, whose treated-group component is missing.

Subtract the largest possible untreated mean from the smallest possible treated mean for the lower ATE bound; reverse those choices for the upper bound. The resulting interval has width one, lies within the logical effect range, and contains zero. It is sharp under these minimal restrictions because the unobserved potential outcomes can be chosen to attain the endpoints without changing the observed distribution. Its width represents ignorance about the counterfactual law, not imprecision from a small observed sample.

E[Y(1)]∈[pμ1,pμ1+1−p],E[Y(0)]∈[(1−p)μ0,(1−p)μ0+p].\mathbb E[Y(1)]\in[p\mu_1,p\mu_1+1-p],\qquad \mathbb E[Y(0)]\in[(1-p)\mu_0,(1-p)\mu_0+p].
Outcome boundedness and consistency identify the observed components; the missing components are bounded without exchangeability.↔ Scroll wide equations horizontally.

Additional assumptions narrow sets but must be named

The bounds are often called worst-case or no-assumptions bounds, but the latter phrase is shorthand: intervention definitions, consistency, outcome support, and the observed population are still specified. Mean independence of potential outcomes from treatment collapses the bounds to the observed mean difference. Monotone treatment response, requiring every unit’s treated outcome to be at least its untreated outcome, instead rules out negative effects while leaving room for partial identification. It is different from monotone selection, which compares baseline prospects across treatment groups.

An identified set presumes knowledge of the observed distribution. With sampled data, its endpoints must be estimated and their uncertainty handled. A plug-in interval between estimated endpoints is not automatically a confidence region for the effect or a simultaneous confidence region for the entire identified set. Those are distinct coverage goals. Statistical uncertainty can shrink as sampling improves while the population identified set remains wide, so a report should distinguish the two rather than calling both an error bar.

τ∈[pμ1−(1−p)μ0−p, pμ1−(1−p)μ0+1−p].\tau\in\left[p\mu_1-(1-p)\mu_0-p,\ p\mu_1-(1-p)\mu_0+1-p\right].
These sharp Robins–Manski bounds concern ATE for outcomes in [0,1], not an ATT-specific sensitivity correction.↔ Scroll wide equations horizontally.

A targeted sensitivity parameter must match the estimand

For ATT define δ₀ as the difference between the treated group’s untreated potential-outcome mean and the observed untreated group’s mean. Add and subtract the first quantity in the observed mean contrast to obtain ATT equal to that contrast minus δ₀. Exchangeability sets this particular bias parameter to zero, while a scientifically chosen range expresses plausible departures. A restriction on δ₀ alone concerns ATT; it does not generally identify ATE, whose treated counterfactual outcome among controls also matters.

Choose the range before inspecting which values preserve a preferred conclusion, and check that it is compatible with the outcome bounds. If the observed control mean is μ₀, the treated group’s untreated mean must remain in [0,1], restricting δ₀ to [−μ₀,1−μ₀]. A sensitivity interval is conditional on that maintained range, rather than a posterior distribution or a frequentist confidence interval by itself. Similar trend-deviation parameters can be used for DID, but the parameter and target must be redefined for that design.

Compute a sharp ATE set

Suppose treatment prevalence is one half, the treated observed outcome mean is 0.8, and the control mean is 0.4. Outcomes are binary and only consistency and no interference are maintained. Derive the ATE bounds.

  1. The treated intervention mean has known component 0.5·0.8=0.4 and missing component between zero and 0.5, yielding [0.4,0.9].

  2. The control intervention mean has known component 0.5·0.4=0.2 and missing component between zero and 0.5, yielding [0.2,0.7].

  3. Subtract endpoints in the appropriate directions: the lower effect is 0.4−0.7=−0.3 and the upper is 0.9−0.2=0.7. Both can be attained by opposite assignments of the missing outcomes.

The result

The observed mean difference 0.4 does not identify the ATE sign under these minimal conditions; the compatible set includes zero and negative values.

Explore the model

Interactive lab

Interactive lab · sampling

More precision does not identify missing counterfactuals

Reset, set success probability 0.4 and sample size 10; predict sample-mean variance 0.4·0.6/10=0.024. Reset, set probability 0.4 and size 100; predict variance 0.0024. Inspect the exact binomial sample-mean distribution, whose playback highlights outcomes rather than drawing data. This illustrates sampling precision for an observed group only; the lab does not shrink a causal identified set. Changes retain time; Reset restores both controls and zero time.

More precision does not identify missing counterfactualsExact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Current readings: Mean sample proportion: 0.5; Sample-mean variance: 0.0125; Uniform CDF error bound: ≤ 0.1062. All parameters are adjustable with the labeled controls below.Exact distribution of the mean of 20 independent Bernoulli variables with success probability 0.5; playback highlights outcome 0.00.250.50.75100.0580.1150.1730.23Sample mean · k / nProbability massNormal bin masses are an approximationk = 0 · exact probability = 9.54e-7
Exact binomial massNormal interval massHighlighted outcome / mean
Mean sample proportion
0.5
Sample-mean variance
0.0125
Uniform CDF error bound
≤ 0.1062
t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forObserved mean uncertainty can shrink while the missing counterfactual component, and thus the identification interval, remains unresolved.

Check your understanding

A range imposed only on untreated selection bias δ₀ directly gives a sensitivity range for which estimand?

Try it yourself

Observed treated and control means are 0.7 and 0.4. Define δ₀=E[Y(0)|A=1]−E[Y(0)|A=0] and assume δ₀ lies between 0.05 and 0.15. Compute the implied ATT range and check compatibility with binary outcome support.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Subtract the larger bias for the lower effect and the smaller bias for the upper effect; check the implied untreated mean among treated units.

Compare with the solution

The observed contrast is 0.3. ATT=0.3−δ₀ therefore ranges from 0.15 to 0.25. The treated group’s untreated mean would be 0.4+δ₀, ranging from 0.45 to 0.55, fully inside [0,1]. The sensitivity restriction is therefore support-compatible. The resulting interval is conditional on the chosen untestable bias range and does not include sampling uncertainty or automatically describe the ATE.

Take it with you

  • Bounded outcomes yield sharp ATE sets even without exchangeability.
  • Identification uncertainty and sampling uncertainty are different and require different reporting.
  • Sensitivity parameters must be defined for the exact estimand and checked against logical support.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.