Propensity scores and overlap
Use a balancing score without confusing good treatment prediction with credible causal comparison.
On this page
The question for today
What you’ll understand
- Derive the true propensity score’s balancing property.
- Separate overlap from the absence of unmeasured confounding.
- Explain why trimming or changing weights can change the target estimand.
The propensity score is the probability of receiving treatment given recorded baseline covariates, not the probability that treatment is beneficial. Its central role is a balancing property: among units sharing the same true score, treatment assignment does not further distinguish the measured covariate distribution. This reduces the dimension of one adjustment problem. It does not certify that the recorded covariates capture all common causes, and a fitted score is not automatically the true probability required by the theorem.
Balancing follows from conditional expectation
Let e(L) equal P(A=1|L). Condition on a score value t. Within that score level, every baseline configuration has the same treatment probability t. Multiplying its conditional probability by t and dividing by the score-level treatment probability leaves its distribution unchanged. The same argument applies to control using one minus t. Thus the distribution of L is identical across treatment arms conditional on the true score, wherever both arms are possible. The theorem is about distributions, not exact pairwise equality of observed covariate vectors.
If treatment is exchangeable given the full baseline covariates, iterated conditioning transfers the required potential-outcome independence to the true score. Consistency and positivity are still needed to identify an effect. A score fitted with a misspecified model can fail to balance even the measured covariates, so balance diagnostics are useful after matching, weighting, or stratification. Those diagnostics assess observed features of L; they cannot establish balance of an unrecorded common cause or prove a causal graph correct.
Overlap is about credible alternatives at the target covariates
For an overall ATE, each covariate region with target probability needs both treatment alternatives. A structural zero or one score means the observed law does not provide one needed outcome regression there. Near-zero or near-one scores preserve formal positivity but can produce unstable comparisons and extreme inverse weights. Strong overlap, bounding scores away from both endpoints, is a useful sufficient condition for many regular inference results. More observations can help with sparse regions, but cannot change a structural impossibility built into treatment assignment.
A treatment prediction model with near-perfect classification may therefore be inconvenient for ATE estimation: it can indicate that comparable alternatives rarely coexist. Prediction accuracy is not the causal design objective. Investigate whether a score reflects genuine assignment restrictions, overly flexible fitting, or a variable set that unnecessarily worsens overlap. Selection of covariates must still respect the identification argument; deleting an important confounder just to improve apparent overlap introduces another problem. Diagnostics illuminate an assumption rather than choosing it automatically.
Changing the comparison population changes the question
Discarding units with extreme scores restricts the population to an overlap region. The resulting effect generally differs from the full-population ATE when effects vary. One should define the retained target and report how it differs from the original population rather than describing trimming as a purely numerical correction. Overlap weights, matching weights, and ATT weights also change the emphasis placed on covariate regions, and need their own estimand interpretation.
For ATT, the treated distribution supplies the target. Its important comparison requirement is that untreated units be available at covariate values occurring among the treated. A region with no treated units does not contribute directly to that target, although it obstructs an overall ATE if it has positive population mass. These support distinctions explain why an effect may be identified for one population and not another. A balancing score is a tool for constructing comparable groups once the target and assumptions have been chosen.
Work it through
Balance a two-stratum population by weighting
A population table has 500 units with L=0 and 500 with L=1. The treated counts are 100 and 400, with control counts 400 and 100. Use true scores 0.2 and 0.8 to calculate the baseline composition after ATE weighting.
Before weighting, L=1 represents 400/500=0.8 of treated units but only 100/500=0.2 of controls. The groups have different baseline mixtures.
Treated inverse-score weights are five for L=0 and 1.25 for L=1. Weighted counts become 100·5=500 and 400·1.25=500.
Control inverse-complement weights are 1.25 for L=0 and five for L=1. Their weighted counts are also 500 and 500, so both weighted arms have baseline proportions one half.
The correct probabilities balance this stipulated population table. Estimating those probabilities from a finite sample introduces additional statistical error.
The living notebook
Explore the model
Interactive lab · network
Extreme scores amplify inverse weights
The sigmoid instrument is only a stipulated propensity-shaped curve. Reset, set input x=−2 and weight w=2; predict score σ(−4.5), about 0.011, and a treated inverse weight near 91. Reset and set x=2,w=2; predict σ(3.5), about 0.971, and an inverse weight near 1.03. The fixed bias is −0.5. Changes keep time; Reset restores both controls and zero time. This model neither estimates a real score nor checks hidden confounding.
- Product · wx
- 1.2
- Logit · wx − 0.5
- 0.7
- Sigmoid output
- 0.6682
One inspectable sigmoid unit, σ(wx − 0.5). Playback traces the computation graph. There is no training, attention mechanism or hidden deep network. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA score close to zero gives a large treated inverse weight even though its fitted probability is mathematically positive.
Pause & reason
Check your understanding
Your turn
Try it yourself
The target population has three baseline strata of masses 0.2,0.5,0.3 and true treatment scores 0,0.3,0.9. Is an unrestricted overall ATE identified by baseline adjustment? Could the zero-score stratum obstruct ATT in the same way?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Ask which target includes units from the zero-score stratum and which potential-outcome mean is unobserved there.
Compare with the solution
The overall ATE is not identified without added extrapolation assumptions because the first stratum has positive population mass but no treated observations, leaving its treated potential-outcome mean unidentified. That stratum contains no treated units, so it has zero weight in the ATT target. The other strata have control probability positive, allowing ATT identification under the remaining consistency and exchangeability assumptions. This does not prove those assumptions or guarantee stable estimation in the stratum with score 0.9.
Keep the idea
Take it with you
- The true propensity score balances measured covariates; a fitted score must be checked.
- Overlap and exchangeability are distinct assumptions.
- Trimming and alternative weights can change the target population and effect.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.