Lesson M10.L05: Differences-in-Differences
| Field | Detail |
|---|---|
| Module | M10 — Instrumental Variables & Causal Inference |
| Level | Graduate (ECMT5001 S1 2027) |
| Duration | 40 min |
| Learning Objective | Derive and apply the 2×2 DiD estimator; state the parallel trends assumption and its limitations; estimate DiD via regression; implement pre-trend testing |
| Provenance | Wooldridge, Introductory Econometrics, Ch 13 (policy analysis); Card & Krueger (1994); MIT 14.32 Recitation Notes (rec_4_13, Spring 2007) |
Explanation
The Evaluation Problem
Suppose a government introduces a new policy — say, a minimum wage increase — in state A but not state B. We want to know: did the policy affect employment?
Naïve comparison (after-only): Compare employment in A vs B after the policy. Problem: the two states may have had different employment levels before the policy, unrelated to the minimum wage.
Naïve comparison (before-after): Compare employment in A before and after. Problem: other factors (national economic conditions, seasonal effects) may have changed simultaneously, confounding the policy effect.
Differences-in-Differences solves both problems by using both dimensions of variation.
The 2×2 DiD Estimator
Consider two groups (treated \(= 1\), control \(= 0\)) and two time periods (pre \(= 0\), post \(= 1\)):
| Pre | Post | Change | |
|---|---|---|---|
| Treated | \(\bar{Y}_{1,0}\) | \(\bar{Y}_{1,1}\) | \(\bar{Y}_{1,1} - \bar{Y}_{1,0}\) |
| Control | \(\bar{Y}_{0,0}\) | \(\bar{Y}_{0,1}\) | \(\bar{Y}_{0,1} - \bar{Y}_{0,0}\) |
The DiD estimator is:
Logic of the two differences:
- First difference (within treated group, pre→post): removes time-invariant characteristics of the treated group (fixed effects).
- Second difference (within control group, pre→post): estimates the counterfactual time trend — what would have happened to the treated group without treatment.
- DiD: the treatment effect = change in treated group minus the trend that would have occurred anyway.
Card & Krueger (1992): Minimum Wage and Employment
One of the most famous natural experiments in economics. New Jersey (NJ) raised its minimum wage from $4.25 to $5.05 in April 1992. Pennsylvania (PA) did not change its minimum wage.
Card & Krueger surveyed fast-food restaurants in NJ and PA before (Feb 1992) and after (Nov 1992) the NJ minimum wage increase.
Employment data (full-time equivalent employees per restaurant):
| Pre (Feb) | Post (Nov) | Change | |
|---|---|---|---|
| NJ (treated) | 20.44 | 21.03 | +0.59 |
| PA (control) | 23.33 | 21.17 | −2.16 |
Interpretation: The minimum wage increase in NJ was associated with approximately 2.75 more full-time equivalent employees per restaurant relative to what would have been expected based on the PA trend. Employment in NJ increased relative to the PA counterfactual — the opposite of the standard competitive labour market prediction.
Why this was controversial
Standard economic theory predicts minimum wages reduce employment by raising labour costs above the competitive equilibrium. Card & Krueger's finding challenged this. The explanation: fast-food restaurants in NJ may have had monopsony power (wage-setting power), so the minimum wage acted more like a correction to a distorted market than a price floor above the competitive wage. This result sparked decades of debate about monopsony in low-wage labour markets.
The Parallel Trends Assumption
The DiD estimate is valid only if:
Parallel Trends: In the absence of treatment, the treated and control groups would have followed parallel trends in the outcome variable.
Formally: \(E(Y_{1,1}^{(0)} - Y_{1,0}^{(0)}) = E(Y_{0,1} - Y_{0,0})\)
where \(Y_{1,1}^{(0)}\) is the counterfactual outcome for the treated group in the post-period without treatment.
This assumption is fundamentally untestable for the post-period — we never observe what would have happened to the treated group without treatment.
However, we can provide indirect evidence:
- Pre-trend test: If multiple pre-treatment periods are available, test whether the two groups had similar trends before treatment. If trends were parallel pre-treatment, it is more credible they would have remained parallel post-treatment.
- Placebo tests: Apply the DiD to a period with no treatment and check that \(\hat{\beta}_{DD} \approx 0\).
When parallel trends may fail
- Mean reversion: Treated units were selected because they were at a temporary low; they would have improved anyway.
- Ashenfelter's dip: Workers who enter training (treatment) had a pre-program earnings dip; their post-program improvement partly reflects mean reversion.
- Different economic conditions: Treated and control regions face different economic shocks in the post-period unrelated to the treatment.
Regression Form of DiD
The DiD can be estimated via regression, allowing control variables and correct standard error computation.
Define: - \(D_i^{treat}\): dummy = 1 if observation belongs to the treated group - \(D_t^{post}\): dummy = 1 for post-treatment period - \(D_i^{treat} \times D_t^{post}\): interaction = 1 for treated group in post-period
The DiD regression is:
Interpretation of each coefficient:
| Coefficient | Meaning |
|---|---|
| \(\alpha\) | Control group mean in pre-period |
| \(\gamma\) | Pre-treatment mean difference: treated − control (pre-period) |
| \(\lambda\) | Time trend: change in control group from pre to post |
| \(\beta\) | DiD estimate = treatment effect |
Verify the 2×2 formula:
- \(\bar{Y}_{0,0} = \alpha\) (control, pre)
- \(\bar{Y}_{0,1} = \alpha + \lambda\) (control, post) → \(\lambda = \bar{Y}_{0,1} - \bar{Y}_{0,0}\)
- \(\bar{Y}_{1,0} = \alpha + \gamma\) (treated, pre)
- \(\bar{Y}_{1,1} = \alpha + \gamma + \lambda + \beta\) (treated, post)
Therefore:
Advantages of the regression approach: - Easy to add control variables \(X_{it}\) (improves precision, doesn't change \(\hat{\beta}\) if controls are balanced) - Clustered standard errors at the group level (accounts for within-group correlation) - Extends naturally to multiple periods and multiple treatment groups
Worked Numerical Example
Setup: A training programme is offered to unemployed workers in region A (treated) in 2022. Region B (control) receives no programme. We measure earnings (\(\$000\)) before (2021) and after (2023).
| Pre (2021) | Post (2023) | Change | |
|---|---|---|---|
| Region A (treated) | 28.4 | 31.2 | +2.8 |
| Region B (control) | 30.1 | 31.4 | +1.3 |
DiD estimate: $\(\hat{\beta}_{DD} = 2.8 - 1.3 = \mathbf{\$1.5\text{k}}\)$
The training programme raised earnings by approximately $1,500 per year, after accounting for the common time trend of $1,300 growth shared by both regions.
Regression form: Regress earnings on \(D^{treat}\), \(D^{post}\), and their interaction. The coefficient on the interaction term should yield \(\hat{\beta} = 1.5\).
Pre-Trend Testing
If data are available for multiple pre-treatment periods (e.g., 2019, 2020, 2021 before a 2022 treatment), we can test parallel trends.
Method: 1. Include "leads" of the treatment dummy: \(D_i^{treat} \times D_{t=-1}^{pre}\), \(D_i^{treat} \times D_{t=-2}^{pre}\), etc. 2. Test whether these pre-treatment interaction coefficients are jointly zero. 3. If zero: the groups had parallel pre-trends → credible parallel trends assumption for the post-period. 4. If non-zero: the groups were already diverging before treatment → DiD estimate may be biased.
Example — Computer-Assisted Learning (Banerjee, Cole, Duflo & Linden 2007, as discussed in MIT 14.32 rec_4_13):
The Pratham/Balsakhi programme randomly assigned computer-assisted learning to some Mumbai schools. The 2×2 DiD was validated by testing whether treated and control schools had identical test score trends before the programme began. The pre-period balance confirmed parallel trends — strengthening the causal interpretation of the post-treatment DiD estimate.
Common Issues
Anticipation effects
If firms or workers change behaviour before the policy takes effect (because they know it's coming), the "pre" period is contaminated. Pre-trend tests may falsely show non-parallel trends when the real cause is anticipation.
Spillovers (SUTVA violation)
If the control group is affected by the treatment (e.g. workers from untreated regions compete with treated workers), the control group is not a valid counterfactual. DiD overestimates the treatment effect.
Compositional changes
If the composition of treated and control groups changes over time (e.g. low-wage workers exit the market after a minimum wage increase), measured employment may not reflect the true policy effect.
Practice Prompts
- A state introduces a carbon tax in 2020. You compare emissions in the taxed state vs a neighbouring untaxed state, before (2018–2019) and after (2021–2022). Write the DiD regression equation and interpret each coefficient.
→ Answer: \(\text{emissions}_{it} = \alpha + \gamma D_i^{tax} + \lambda D_t^{post} + \beta(D_i^{tax} \times D_t^{post}) + u_{it}\). \(\alpha\) = untaxed state mean emissions pre-2020; \(\gamma\) = pre-policy emission difference (taxed − untaxed); \(\lambda\) = trend in untaxed state; \(\beta\) = policy effect (expected negative if the carbon tax reduces emissions).
- Pre-trend test: You have emissions data for 2015–2019 (pre-policy) and 2020–2022 (post-policy). How would you test the parallel trends assumption?
→ Answer: Include interaction terms \(D_i^{tax} \times D_t^{2016}\), \(D_i^{tax} \times D_t^{2017}\), \(D_i^{tax} \times D_t^{2018}\), \(D_i^{tax} \times D_t^{2019}\) in the regression (with 2015 as base year). If these pre-policy interaction coefficients are jointly insignificant (F-test), parallel trends hold. If they are significant, the two states were already diverging before the tax — the DiD estimate is unreliable.
- The Card & Krueger DiD gives \(\hat{\beta}_{DD} = 2.75\). What additional evidence would strengthen the causal interpretation?
→ Answer: (a) Pre-trend test showing NJ and PA had parallel employment trends before the minimum wage change; (b) placebo tests using other periods or other industries not affected by the minimum wage; (c) robustness to alternative control groups; (d) showing the effect is larger in restaurants that were most affected (below the new minimum wage before the policy) — a "dose-response" pattern consistent with causality.
Further Resources
- 📚 Wooldridge, Ch 13 (Policy Analysis using Pooled Cross-Sections and Panel Data)
- 📚 Card, D. & Krueger, A.B. (1994) "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania." American Economic Review 84(4): 772–793.
- 📚 Angrist, J. & Pischke, J.S. (2009) Mostly Harmless Econometrics, Ch 5 (Parallel Worlds)
- 📺 DiD Explained — Nick Huntington-Klein (20 min)
- 📚 MIT 14.32 Recitation Notes — rec_4_13 (Spring 2007): "More on Differences-in-Differences" — Card & Krueger 2×2 matrix, parallel trends testing, Banerjee et al. computer-assisted learning RCT