7 Matching Estimators
Matching is a way to make treated and control observations more comparable. For each treated observation, we look for controls with similar covariates. Then we compare outcomes in this matched sample.
This does not solve endogeneity. Matching only helps with selection on observables. If the treatment is selected on unobserved variables, matching will not fix the problem. The value of matching is that it makes the overlap problem visible. A regression can extrapolate quietly; matching often shows that there are no good controls for part of the treated sample.
In R, the main workflow is MatchIt for matching and cobalt for balance checking.
Related reading: A longer treatment is in Matching and Weighting Part 1 of Topics on Econometrics and Causal Inference, which credits Noah Greifer and coauthors (the
MatchIt/WeightItpackage authors).
7.1 Assumptions
Matching needs the same assumptions as other selection-on-observables methods:
- SUTVA — no interference, no hidden treatment versions.
- Ignorability (unconfoundedness) — conditional on \(X\), the treatment \(D\) is independent of the potential outcomes \((Y(0), Y(1))\).
- Overlap (positivity) — every value of \(X\) has positive probability of being both treated and untreated.
Even if ignorability is true, the covariate distribution can be very different in the treated and control groups. Matching tries to repair that before estimating the effect. If there is no overlap, it should not pretend there is overlap.
7.2 The Lalonde example
I use the Lalonde job-training data because it is the standard example for matching. The treatment is job training. The outcome is 1978 earnings. The data set in MatchIt is the observational version, so the treated and control groups are not balanced at the start.
Rows: 614
Columns: 9
$ treat <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1…
$ age <int> 37, 22, 30, 27, 33, 22, 23, 32, 22, 33, 19, 21, 18, 27, 17, 1…
$ educ <int> 11, 9, 12, 11, 8, 9, 12, 11, 16, 12, 9, 13, 8, 10, 7, 10, 13,…
$ race <fct> black, hispan, black, black, black, black, black, black, blac…
$ married <int> 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0…
$ nodegree <int> 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1…
$ re74 <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ re75 <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ re78 <dbl> 9930.0460, 3595.8940, 24909.4500, 7506.1460, 289.7899, 4056.4…There are 614 observations, 185 treated and 429 control. The treatment is treat; the outcome is re78, 1978 earnings in dollars; the covariates are age, educ (years of schooling), race (black, hispanic, white), married, nodegree (no high-school degree), and re74 and re75, earnings in 1974 and 1975. The earnings variables matter most here, because they pile up at zero and differ sharply between the groups.
Before matching anything we check balance. bal.tab reports the standardized mean difference for each covariate and flags any above 0.1:
Code
Balance Measures
Type Diff.Un M.Threshold.Un
distance Distance 1.7941
age Contin. -0.3094 Not Balanced, >0.1
educ Contin. 0.0550 Balanced, <0.1
race_black Binary 0.6404 Not Balanced, >0.1
race_hispan Binary -0.0827 Balanced, <0.1
race_white Binary -0.5577 Not Balanced, >0.1
married Binary -0.3236 Not Balanced, >0.1
nodegree Binary 0.1114 Not Balanced, >0.1
re74 Contin. -0.7211 Not Balanced, >0.1
re75 Contin. -0.2903 Not Balanced, >0.1
Balance tally for mean differences
count
Balanced, <0.1 2
Not Balanced, >0.1 7
Variable with the greatest mean difference
Variable Diff.Un M.Threshold.Un
re74 -0.7211 Not Balanced, >0.1
Sample sizes
Control Treated
All 429 185Seven of the nine contrasts exceed 0.1. Earnings in 1974 differ by \(-0.72\) standard deviations, the black indicator by 0.64, the white indicator by \(-0.56\), marital status by \(-0.32\), age by \(-0.31\), earnings in 1975 by \(-0.29\). Only educ (0.055) and the hispanic indicator (\(-0.083\)) pass. The propensity score itself differs by 1.79 standard deviations. This sample is nowhere near balanced.
7.3 Distance measures
Matching needs a distance metric. The usual choices are:
- Propensity score: estimate \(\hat e(x)=P(D=1 \mid X=x)\) and match on the fitted probability. This turns many covariates into one score.
- Mahalanobis distance: match on the raw covariates using the squared distance \((x_i-x_j)'\Sigma^{-1}(x_i-x_j)\) (rankings are unchanged by the square root). This works better in low dimension.
- Hybrid: impose a propensity-score caliper, then use Mahalanobis distance inside the caliper.
7.4 Matching methods
7.4.1 Nearest-neighbour matching on a propensity score
We fit the propensity score by logistic regression, take its linear index, and match each treated unit to the single closest control, without replacement.
Code
m.nn <- matchit(treat ~ age + educ + race + married + nodegree + re74 + re75,
data = lalonde,
method = "nearest",
distance = "glm",
link = "linear.logit",
ratio = 1)
m.nnA `matchit` object
- method: 1:1 nearest neighbor matching without replacement
- distance: Propensity score
- estimated with logistic regression and linearized
- number of obs.: 614 (original), 370 (matched)
- target estimand: ATT
- covariates: age, educ, race, married, nodegree, re74, re75All 185 treated units are matched, so 370 of the 614 observations are used and 244 controls are discarded. The target estimand is the ATT.
Now check balance again:
Balance Measures
Type Diff.Adj M.Threshold
distance Distance 0.9192
age Contin. 0.0718 Balanced, <0.1
educ Contin. -0.1290 Not Balanced, >0.1
race_black Binary 0.3730 Not Balanced, >0.1
race_hispan Binary -0.1568 Not Balanced, >0.1
race_white Binary -0.2162 Not Balanced, >0.1
married Binary -0.0216 Balanced, <0.1
nodegree Binary 0.0703 Balanced, <0.1
re74 Contin. -0.0505 Balanced, <0.1
re75 Contin. -0.0257 Balanced, <0.1
Balance tally for mean differences
count
Balanced, <0.1 5
Not Balanced, >0.1 4
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
race_black 0.373 Not Balanced, >0.1
Sample sizes
Control Treated
All 429 185
Matched 185 185
Unmatched 244 0Five of nine now pass, up from two. The earnings histories are repaired: re74 goes from \(-0.72\) to \(-0.05\) and re75 from \(-0.29\) to \(-0.03\). Race is not: the black indicator is still 0.373 and the white indicator \(-0.216\). And educ has got worse, from 0.055 to \(-0.129\). This is what matching on a scalar score does: it balances the score, not the covariates. Two units with the same score can differ on individual covariates in offsetting ways, and nothing in the procedure prevents that.
7.4.2 Full matching
Full matching forms subclasses with treated and control observations inside each subclass. It can put one treated unit with several controls, or one control with several treated units. It uses all observations and then assigns weights.
Code
Balance Measures
Type Diff.Adj M.Threshold
distance Distance 0.0045 Balanced, <0.1
age Contin. 0.0393 Balanced, <0.1
educ Contin. -0.0956 Balanced, <0.1
race_black Binary 0.0043 Balanced, <0.1
race_hispan Binary 0.0103 Balanced, <0.1
race_white Binary -0.0146 Balanced, <0.1
married Binary 0.0259 Balanced, <0.1
nodegree Binary 0.0504 Balanced, <0.1
re74 Contin. -0.0009 Balanced, <0.1
re75 Contin. -0.0091 Balanced, <0.1
Balance tally for mean differences
count
Balanced, <0.1 10
Not Balanced, >0.1 0
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
educ -0.0956 Balanced, <0.1
Sample sizes
Control Treated
All 429. 185
Matched (ESS) 50.76 185
Matched (Unweighted) 429. 185All ten contrasts now pass, the largest being educ at \(-0.096\), and no observation is discarded. The cost is in the weights: the control effective sample size falls from 429 to 50.8, because full matching concentrates weight on the controls that resemble treated units.
The effective sample size is the number of equally weighted observations that would carry the same precision. An unweighted mean of \(m\) observations has variance \(\sigma^2/m\); a weighted mean with weights summing to one has variance \(\sigma^2 \sum_i w_i^2\). Setting the two equal gives \(m = 1/\sum_i w_i^2\). So 50.8 means these 429 controls carry about as much information as 51 equally weighted ones. Balance improved and precision did not.
7.4.3 Mahalanobis matching
When the number of covariates is small, it is reasonable to match directly on the covariates with Mahalanobis distance.
Code
Balance Measures
Type Diff.Adj M.Threshold
age Contin. 0.1269 Not Balanced, >0.1
educ Contin. -0.0430 Balanced, <0.1
race_black Binary 0.3784 Not Balanced, >0.1
race_hispan Binary 0.0000 Balanced, <0.1
race_white Binary -0.3784 Not Balanced, >0.1
married Binary -0.0595 Balanced, <0.1
nodegree Binary 0.0486 Balanced, <0.1
re74 Contin. -0.2476 Not Balanced, >0.1
re75 Contin. -0.1322 Not Balanced, >0.1
Balance tally for mean differences
count
Balanced, <0.1 4
Not Balanced, >0.1 5
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
race_black 0.3784 Not Balanced, >0.1
Sample sizes
Control Treated
All 429 185
Matched 185 185
Unmatched 244 0Mahalanobis matching balances only four of nine. The black and white indicators sit at \(\pm 0.378\), re74 at \(-0.248\), re75 at \(-0.132\), and age at 0.127. It does worse than the propensity score did on earnings. With a three-level race factor and two earnings variables that pile up at zero, the covariance matrix is a poor description of the covariate space, and the metric it defines is a poor guide to who resembles whom.
7.4.4 Coarsened Exact Matching (CEM)
CEM coarsens covariates into bins and then exact-matches on the binned values. This is very transparent: if a treated observation has no comparable control in the binned covariate space, it is dropped.
Code
Balance Measures
Type Diff.Adj M.Threshold
age Contin. 0.0493 Balanced, <0.1
educ Contin. 0.0446 Balanced, <0.1
race_black Binary 0.0000 Balanced, <0.1
race_hispan Binary 0.0000 Balanced, <0.1
race_white Binary 0.0000 Balanced, <0.1
married Binary 0.0000 Balanced, <0.1
nodegree Binary 0.0000 Balanced, <0.1
re74 Contin. -0.0427 Balanced, <0.1
re75 Contin. -0.0492 Balanced, <0.1
Balance tally for mean differences
count
Balanced, <0.1 9
Not Balanced, >0.1 0
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
age 0.0493 Balanced, <0.1
Sample sizes
Control Treated
All 429. 185
Matched (ESS) 41.29 65
Matched (Unweighted) 75. 65
Unmatched 354. 120All nine contrasts pass, and the largest is age at 0.049. Read the sample sizes before celebrating: only 65 of the 185 treated units are matched, and 120 are dropped, along with 354 of the 429 controls. The control effective sample size is 41.3.
CEM can drop many observations. That is not necessarily a problem. It is often telling us that the original sample does not support the target comparison. But it does change the question. The estimand is now the average effect for the 65 treated units that had a comparable control, not the ATT for all 185, and those two need not be the same number.
7.5 Estimation after matching
After matching, we estimate the effect on the matched data. The matching weights come from MatchIt. With subclass matching, standard errors should be clustered by subclass: units within the same matched subclass are not independent draws – they were selected together specifically because they resemble each other on the matching covariates, and (for many-to-one or full matching) a single unit’s outcome can appear, re-weighted, in the “observation” for more than one comparison. Treating them as independent (ordinary or heteroskedasticity-robust SEs) understates the true sampling variability; clustering by subclass accounts for that within-subclass correlation.
Code
m.data <- match_data(m.full)
fit <- lm(re78 ~ treat * (age + educ + race + married + nodegree + re74 + re75),
data = m.data,
weights = weights)
# Use marginaleffects for the average treatment effect with cluster-by-subclass SEs
avg_comparisons(fit,
variables = "treat",
vcov = ~subclass,
newdata = subset(m.data, treat == 1)) # ATT
Estimate Std. Error z Pr(>|z|) S 2.5 % 97.5 %
1977 704 2.81 0.00501 7.6 596 3357
Term: treat
Type: response
Comparison: 1 - 0The full-matching ATT is $1,977 with a subclass-clustered standard error of $704, a 95% interval of \([596, 3357]\), and \(p = 0.005\). Training raised 1978 earnings for the trained.
The point is not that matching mechanically produces the “right” answer. The point is that after improving balance, the estimate is much less driven by obvious covariate differences.
7.6 Balance plots
cobalt::love.plot is the easiest way to see the balance change. It plots standardized mean differences before and after matching.
Code
Each variable appears twice, unadjusted and adjusted, with the variables ordered by their unadjusted difference and differences plotted in absolute value. Every point moves toward zero except educ, which starts at 0.055 and ends at 0.096 — it was already balanced, and full matching traded a little of it for the rest. re74, the worst offender before matching at \(-0.72\), moves furthest, ending at \(-0.0009\).
A useful rule is that standardized mean differences should be below 0.1, or below 0.05 if we want to be stricter.
7.7 Matching with replacement vs without
By default a control observation can be used only once. With replacement, the same good control can be used several times. This often improves balance when overlap is weak, but the effective sample size becomes smaller.
Code
Balance Measures
Type Diff.Adj M.Threshold
distance Distance 0.0044 Balanced, <0.1
age Contin. 0.2395 Not Balanced, >0.1
educ Contin. -0.0161 Balanced, <0.1
race_black Binary 0.0054 Balanced, <0.1
race_hispan Binary -0.0054 Balanced, <0.1
race_white Binary 0.0000 Balanced, <0.1
married Binary 0.0595 Balanced, <0.1
nodegree Binary 0.0054 Balanced, <0.1
re74 Contin. -0.0493 Balanced, <0.1
re75 Contin. 0.0087 Balanced, <0.1
Balance tally for mean differences
count
Balanced, <0.1 9
Not Balanced, >0.1 1
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
age 0.2395 Not Balanced, >0.1
Sample sizes
Control Treated
All 429. 185
Matched (ESS) 46.31 185
Matched (Unweighted) 82. 185
Unmatched 347. 0With replacement, nine of ten contrasts pass, against five of nine without. age is the exception, at 0.240. Only 82 distinct controls are used, and their effective sample size is 46.3. Compared with the 1:1 match without replacement, which used 185 distinct controls, the same good controls are being reused.
If matching without replacement cannot balance the sample, matching with replacement is a reasonable next step.
7.8 ATE, ATT, ATC — choose carefully
MatchIt usually targets the ATT. That means we are asking what treatment did for the treated units. For ATE, use estimand = "ATE" and a method that keeps the whole sample, such as full matching or weighting. For ATC, use estimand = "ATC". These are not just software options; they are different causal questions.
7.9 When matching fails
Matching fails in predictable ways:
- High-dimensional matching is hard. Propensity scores reduce dimension, but they do not create overlap.
- If some values of \(X\) have no treated or no control units, no matching method can recover the missing comparison.
- Propensity-score matching depends on the propensity-score model. If that model is bad, the matching can be bad too.
- Matching does not address hidden confounding. Rosenbaum bounds, discussed in the Sensitivity Analysis chapter, ask how strong hidden bias would have to be to change the conclusion.
7.10 Matching vs weighting
Matching and weighting do similar jobs. Matching chooses comparable observations. Weighting keeps observations but changes their contribution so the treated and control covariate distributions become closer.
When overlap is good, both approaches often give similar answers. When overlap is bad, both approaches become fragile. The honest choices then are to change the target population, to report the lack of overlap plainly, to move to overlap weights or another explicitly chosen target, or to trim – accepting that trimming changes the estimand rather than rescuing the original one.
Doubly-robust estimators do not belong on that list. AIPW and TMLE protect against misspecification of the nuisance models, and they still require the treatment probabilities to be bounded away from zero, and away from one for an ATE. Where support is genuinely missing the inverse weights are extreme, and a doubly-robust estimator is then less stable than a simple one, not more. Robustness to a wrong model is not robustness to a missing comparison.
7.11 Weighting Alternatives
WeightIt implements several weighting methods. The companion blog chapter on weighting has more detail. Here I use the same Lalonde data so the comparison is easy.
7.11.1 Inverse probability of treatment weighting (IPW)
The baseline method is IPW. Estimate the propensity score, then weight by the inverse probability of receiving the observed treatment.
Code
Balance Measures
Type Diff.Adj M.Threshold
prop.score Distance -0.0205 Balanced, <0.05
age Contin. 0.1188 Not Balanced, >0.05
educ Contin. -0.0284 Balanced, <0.05
race_black Binary -0.0022 Balanced, <0.05
race_hispan Binary 0.0002 Balanced, <0.05
race_white Binary 0.0021 Balanced, <0.05
married Binary 0.0186 Balanced, <0.05
nodegree Binary 0.0184 Balanced, <0.05
re74 Contin. -0.0021 Balanced, <0.05
re75 Contin. 0.0110 Balanced, <0.05
Balance tally for mean differences
count
Balanced, <0.05 9
Not Balanced, >0.05 1
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
age 0.1188 Not Balanced, >0.05
Effective sample sizes
Control Treated
Unadjusted 429. 185
Adjusted 99.82 185Note the stricter 0.05 threshold from here on. IPW balances nine of ten; age fails at 0.119. The control effective sample size is 99.8 out of 429, better than any of the matching methods above.
7.11.2 Covariate balancing propensity score (CBPS)
CBPS estimates the propensity score while also trying to balance covariate means. So the propensity-score model is not judged only by likelihood; it is also judged by balance.
Code
Balance Measures
Type Diff.Adj M.Threshold
prop.score Distance -0.018 Balanced, <0.05
age Contin. 0.000 Balanced, <0.05
educ Contin. -0.000 Balanced, <0.05
race_black Binary -0.000 Balanced, <0.05
race_hispan Binary -0.000 Balanced, <0.05
race_white Binary 0.000 Balanced, <0.05
married Binary -0.000 Balanced, <0.05
nodegree Binary -0.000 Balanced, <0.05
re74 Contin. -0.000 Balanced, <0.05
re75 Contin. -0.000 Balanced, <0.05
Balance tally for mean differences
count
Balanced, <0.05 10
Not Balanced, >0.05 0
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
re74 -0 Balanced, <0.05
Effective sample sizes
Control Treated
Unadjusted 429. 185
Adjusted 98.46 185Every covariate mean difference is zero to three decimals, and the control effective sample size is 98.5.
7.11.3 Entropy balancing
Entropy balancing chooses weights so that the weighted control group matches the treated group on covariate means. It is useful because the balance constraint is explicit.
Code
Balance Measures
Type Diff.Adj M.Threshold
age Contin. -0 Balanced, <0.05
educ Contin. -0 Balanced, <0.05
race_black Binary 0 Balanced, <0.05
race_hispan Binary 0 Balanced, <0.05
race_white Binary -0 Balanced, <0.05
married Binary -0 Balanced, <0.05
nodegree Binary 0 Balanced, <0.05
re74 Contin. -0 Balanced, <0.05
re75 Contin. -0 Balanced, <0.05
Balance tally for mean differences
count
Balanced, <0.05 9
Not Balanced, >0.05 0
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
re74 -0 Balanced, <0.05
Effective sample sizes
Control Treated
Unadjusted 429. 185
Adjusted 98.46 185The standardized mean differences are essentially zero because entropy balancing imposes mean balance directly. The effective sample size is 98.5.
That is the same table CBPS produced, and it is not a coincidence: the two sets of control weights are numerically identical here, agreeing to twelve decimal places. Just-identified CBPS for the ATT solves the exact mean-balance moment conditions with a logistic link, which makes the control weights proportional to \(\exp(x'\lambda)\). Entropy balancing minimises the Kullback-Leibler divergence from uniform weights subject to the same exact mean-balance constraints, and its solution has the same \(\exp(x'\lambda)\) form. Same constraints, same functional form, same weights. The two methods are motivated differently and arrive at one answer.
7.11.4 Energy balancing
Energy balancing targets the whole covariate distribution, not just means. It is more ambitious than entropy balancing, and usually more computationally expensive.
Code
Balance Measures
Type Diff.Adj M.Threshold
age Contin. -0.0016 Balanced, <0.05
educ Contin. 0.0106 Balanced, <0.05
race_black Binary 0.0060 Balanced, <0.05
race_hispan Binary -0.0008 Balanced, <0.05
race_white Binary -0.0053 Balanced, <0.05
married Binary -0.0011 Balanced, <0.05
nodegree Binary 0.0050 Balanced, <0.05
re74 Contin. -0.0021 Balanced, <0.05
re75 Contin. 0.0226 Balanced, <0.05
Balance tally for mean differences
count
Balanced, <0.05 9
Not Balanced, >0.05 0
Variable with the greatest mean difference
Variable Diff.Adj M.Threshold
re75 0.0226 Balanced, <0.05
Effective sample sizes
Control Treated
Unadjusted 429. 185
Adjusted 41.82 185All nine contrasts pass, the largest being re75 at 0.023 — close to zero but not exactly zero, because energy balancing does not impose mean balance as a constraint. The control effective sample size is 41.8, less than half of entropy balancing’s 98.5. That is the price of targeting the whole covariate distribution rather than its means.
7.11.5 Estimation with weighted-aware regression
After computing weights, use lm_weightit() instead of plain lm if we want standard errors that account for the estimated weights.
Code
fit_ebal <- lm_weightit(
re78 ~ treat * (age + educ + race + married + nodegree + re74 + re75),
data = lalonde,
weightit = w_ebal
)
avg_comparisons(fit_ebal, variables = "treat",
newdata = subset(lalonde, treat == 1)) # ATT
Estimate Std. Error z Pr(>|z|) S 2.5 % 97.5 %
1273 770 1.65 0.0983 3.3 -236 2783
Term: treat
Type: probs
Comparison: 1 - 0The entropy-balanced ATT is $1,273 with a sandwich standard error of $770, a 95% interval of \([-236, 2783]\), and \(p = 0.098\). The interval includes zero.
7.11.6 When to use each weighting method
| Method | Strengths | Weaknesses |
|---|---|---|
| IPW (glm) | Familiar; well-studied | Sensitive to PS misspecification; extreme weights |
| CBPS | Balance-constrained PS; robust to model misspecification | Slower; can fail with many covariates |
| Entropy balancing | Exact mean balance; doubly robust for ATT | Balances means only, not distributions |
| Energy balancing | Balances entire distribution | Computationally heavier; less mature theory |
For applied work, entropy balancing is often a good default because it is simple and the balance diagnostics are easy to explain. Energy balancing is useful when matching the whole covariate distribution is important.
7.11.7 Comparing matching to weighting
The matching and weighting examples here target the same ATT, so put the two estimates next to each other rather than reporting whichever came last:
Code
cmp <- rbind(
transform(as.data.frame(
avg_comparisons(fit, variables = "treat", vcov = ~subclass,
newdata = subset(m.data, treat == 1))),
method = "Full matching (subclass-clustered SE)"),
transform(as.data.frame(
avg_comparisons(fit_ebal, variables = "treat",
newdata = subset(lalonde, treat == 1))),
method = "Entropy balancing (sandwich SE)")
)
cmp[, c("method", "estimate", "std.error", "conf.low", "conf.high", "p.value")] |>
knitr::kable(digits = c(0, 0, 0, 0, 0, 3),
caption = "Two routes to the same ATT on the Lalonde data")| method | estimate | std.error | conf.low | conf.high | p.value |
|---|---|---|---|---|---|
| Full matching (subclass-clustered SE) | 1977 | 704 | 596 | 3357 | 0.005 |
| Entropy balancing (sandwich SE) | 1273 | 770 | -236 | 2783 | 0.098 |
The gap is instructive. The two point estimates differ by roughly one standard error, and one interval excludes zero while the other does not — a reader who sees only one of these tables would draw a different conclusion about whether the programme worked. Note also that the ranking on balance does not settle it: entropy balancing imposes exact mean balance (every standardized difference above is at most 0.023), whereas full matching leaves educ at \(-0.096\), yet it is entropy balancing that gives the smaller estimate and the wider interval. Better mean balance is not the same thing as a more precise or more credible answer.
This is model dependence, not a bug in either method. In an applied paper I would report balance for each method, not just the final coefficient, and I would report both estimates. If several reasonable methods give similar balance and similar estimates, the result is more credible. If they differ sharply, as here, the problem is usually weak overlap or model dependence, and the honest summary says so instead of picking the number that crosses the significance line.
For more details on MatchIt and the Stata teffects commands, see the matching blog chapter and treatment-effects in Stata.
7.12 Matching, weighting, and direct estimation
The estimation chapter estimated a treatment effect by regression adjustment, IPW, AIPW and IPWRA. This chapter estimated one by matching and by four weighting methods. Those are not two families of estimators. They are one estimand and several ways of building it, and it is worth saying how they line up.
The ATT is
\[ \tau_{ATT} = E[Y \mid W = 1] - E_{X \mid W = 1}\bigl[\, E[Y \mid W = 0, X] \,\bigr]. \tag{7.1}\]
The first term is observed. It is the mean outcome among the treated. The second term is not observed. It is the mean outcome the treated would have had untreated, averaged over the covariate distribution of the treated. Every method in both chapters is a way of forming that second term, and every one of them ends up as a weighted average of control outcomes, \(\sum_{i: W_i = 0} w_i Y_i\). What differs is where the weights come from.
| Route | What is modelled | Weight on control \(i\) |
|---|---|---|
| Regression adjustment | the outcome, \(\mu_0(x) = E[Y \mid W=0, X=x]\) | implicit in the fitted values used to predict at treated \(x\) |
| IPW | the propensity score \(\pi(x)\) | the odds \(\pi(x_i)/(1-\pi(x_i))\) |
| Matching | neither; a distance rule | 0 if unmatched, else 1, the number of times reused, or the subclass ratio |
| Entropy, CBPS, energy balancing | the weights themselves | whatever satisfies the balance constraints |
| AIPW | both | the odds, applied to the residual \(Y - \hat\mu_0(X)\) |
Weighting is IPW. That is the second row, and weightit with estimand = "ATT" and method = "glm" returns exactly those odds. CBPS, entropy balancing and energy balancing do not change the estimator. They change how the weights are found. IPW fits \(\pi\) by maximum likelihood and then transforms it into weights; the balancing methods skip the likelihood and solve for the weights directly. For entropy balancing the link is exact rather than an analogy: the control weights are proportional to \(\exp(x'\lambda)\), as the CBPS comparison above showed, and that is the odds of a logistic propensity score. Entropy balancing is IPW for a logit model whose coefficients are set by exact mean balance instead of by likelihood.
Matching is weighting too, with the weights restricted to zero and small integers and chosen by a distance rule rather than by a model. That is what lets match_data hand a weights column to lm. The restriction is where the trade-off lives. Matching sets some weights to zero, so it discards observations, and when it discards treated units — as CEM did above, keeping 65 of 185 — the estimand becomes the ATT among the matched treated. Weighting keeps everyone, so the estimand survives, and pays for it with extreme weights when overlap is poor, which is what the effective sample sizes in this chapter have been reporting. Matching buys stability with the estimand. Weighting buys the estimand with variance.
AIPW uses both columns. Start from regression adjustment, then add the odds-weighted average of the control residuals \(Y - \hat\mu_0(X)\). It is consistent if either model is right. The section on estimation after matching did an informal version of the same thing: match first, then run a regression with treatment-covariate interactions on the matched sample. That is regression adjustment after matching, and the regression plays the part of the augmentation term. It is a relative of Abadie and Imbens (2011) bias-corrected matching rather than the same estimator: their correction is applied unit by unit to matched outcomes and comes with its own variance theory, neither of which a post-matching interacted regression reproduces.
Five routes, one data set. Each row below is built by hand from the weight representation so the arithmetic is visible.
Code
covs <- c("age", "educ", "race", "married", "nodegree", "re74", "re75")
trt <- lalonde$treat == 1
y <- lalonde$re78
# Outcome model, fitted on controls only, then predicted everywhere
mu0_fit <- lm(reformulate(covs, response = "re78"),
data = subset(lalonde, treat == 0))
mu0 <- predict(mu0_fit, newdata = lalonde)
# Odds weights from a logistic propensity score
ps <- predict(glm(reformulate(covs, response = "treat"),
data = lalonde, family = binomial), type = "response")
odds <- ps / (1 - ps)
wt_ipw <- ifelse(trt, 1, odds)
# Weighted difference in means: the common shape of rows 2 to 4
hajek <- function(w) {
weighted.mean(y[trt], w[trt]) - weighted.mean(y[!trt], w[!trt])
}
att_ra <- mean(y[trt] - mu0[trt])
att_ipw <- hajek(wt_ipw)
att_eb <- hajek(w_ebal$weights)
att_fm <- hajek(m.full$weights)
att_aipw <- att_ra - weighted.mean(y[!trt] - mu0[!trt], odds[!trt])
data.frame(
route = c("Regression adjustment (outcome model)",
"IPW (propensity model)",
"Entropy balancing (balance constraints)",
"Full matching (distance rule)",
"AIPW (both models)"),
att = c(att_ra, att_ipw, att_eb, att_fm, att_aipw)
) |>
knitr::kable(digits = 0, col.names = c("Route", "ATT"),
caption = "Five routes to the same ATT on the Lalonde data.")| Route | ATT |
|---|---|
| Regression adjustment (outcome model) | 1648 |
| IPW (propensity model) | 1214 |
| Entropy balancing (balance constraints) | 1273 |
| Full matching (distance rule) | 1855 |
| AIPW (both models) | 1231 |
Two of these numbers confirm the algebra directly. The hand-built odds weights agree with the weightit propensity-score weights from the IPW section above to machine precision, so the claim that weighting is IPW is an identity here and not a resemblance. And entropy balancing gives $1,273 — the same figure that lm_weightit produced earlier with the full set of treatment-covariate interactions, agreeing to nine decimal places. Exact mean balance leaves nothing for a linear outcome model to correct, so adding one changes the estimate not at all. That is the double robustness of entropy balancing (Zhao and Percival 2017) appearing as arithmetic: the same weights deliver the IPW estimator for a logit propensity model and the regression-adjustment estimator for a linear outcome model.
The spread across routes is the part to read carefully. Regression adjustment gives $1,648 and IPW gives $1,214. AIPW starts from the regression-adjustment figure and the augmentation term subtracts $417, landing at $1,231 — next to IPW, not next to regression adjustment. The propensity side is doing the work here, which is what we should expect when the treated and control covariate distributions are as far apart as the balance tables in this chapter have shown. Full matching sits highest at $1,855. It fits a propensity score too — a probit, used as the distance for forming subclasses — so the gap between it and IPW’s $1,214 is not about whether a propensity model was estimated. It is about how the score is turned into weights: coarse subclass weights that throw away the score’s fine gradations, against the odds transformation that keeps them.
These are point estimates. Standard errors are not comparable across the rows and are deliberately left out: matching is not a smooth functional of the data, so the bootstrap is invalid for it (Abadie and Imbens 2008) and the subclass clustering used earlier is needed, while IPW and AIPW have influence-function variances and AIPW attains the semiparametric efficiency bound that fixed-\(M\) matching does not — given positivity, enough regularity, nuisance estimators converging fast enough (with cross-fitting when they are flexible), and both nuisance limits at the truth. The bound is a property of the estimator plus those conditions, not of the name. The lesson from the column of estimates is the same one the matching-versus-weighting comparison gave: with overlap this weak, the choice of route moves the answer by more than one standard error, so report several routes rather than the one that reads best.
7.13 Summary
- Matching is for selection on observables. It does not fix unobserved confounding.
- The main diagnostic is covariate balance, not the treatment-effect coefficient.
- Propensity-score matching is simple, but full matching or weighting often balances better.
- If there is no overlap, change the estimand or restrict the sample. Do not hide the support problem.
- Matching, weighting, and the direct estimators of the previous chapter are one estimand built several ways. Weighting with
method = "glm"is IPW, and matching is weighting with the weights restricted to zero and small integers.