Lesson M07.L03: Missing Data, Sample Selection, and Outliers

Field	Detail
Module	07 — Specification Issues
Level	Graduate (ECMT5001 S1 2027)
Duration	30–35 min
Learning Objective	Classify missing data mechanisms and understand when listwise deletion is valid; recognise sample selection bias; distinguish truncation from censoring; identify outliers using leverage and Cook's distance.
Provenance	Wooldridge, Introductory Econometrics, Ch 9.4

Explanation

1. Missing Data Mechanisms

Missing data is pervasive in applied work. The consequences for OLS depend critically on why observations are missing. Rubin's (1976) taxonomy identifies three mechanisms:

MCAR — Missing Completely At Random

Missingness is unrelated to any variable in the dataset, observed or unobserved:

\[P(\text{missing} \mid y, x_1, \ldots, x_k) = P(\text{missing})\]

Example: A survey firm loses 5% of questionnaires due to a server failure unrelated to respondents' characteristics.

Consequence: The observed sample is a random sub-sample of the full population. OLS on complete cases is unbiased and consistent — you just have a smaller $n$ (and thus larger standard errors).

MAR — Missing At Random

Missingness depends on observed variables but not on the missing value itself, conditional on observables:

\[P(\text{missing} \mid y, x_1, \ldots, x_k) = P(\text{missing} \mid x_1, \ldots, x_k)\]

Example: Higher-income respondents are less likely to report their wealth (income is observed; wealth is missing), but the probability of missing wealth does not further depend on the actual (unobserved) wealth level conditional on income.

Consequence: Complete-case OLS may be biased but modern methods (multiple imputation, inverse probability weighting) can recover consistent estimates.

MNAR — Missing Not At Random

Missingness depends on the unobserved value itself:

\[P(\text{missing} \mid y, x_1, \ldots, x_k) \text{ depends on } y \text{ even conditional on } x_1, \ldots, x_k\]

Example: People with very high (or very low) income refuse to disclose it — the missingness is driven by the income value itself.

Consequence: Complete-case OLS is biased and inconsistent. This is the hardest case; requires strong modelling assumptions or additional data.

2. Complete-Case Analysis (Listwise Deletion)

Listwise deletion (dropping all rows with any missing value) is the default in most software. It is:

Valid (unbiased) under MCAR.
Potentially valid but inefficient under MAR (if you don't model the selection).
Biased under MNAR.

Practical guidance: Before dropping rows, test whether missingness is correlated with observed $y$ or $x$ variables. Regress a missingness indicator on the covariates; significant coefficients suggest MAR or MNAR, not MCAR.

3. Sample Selection Problem

Sample selection bias arises when the observed sample is systematically different from the population of interest, and this selection is correlated with the outcome.

Classic example — wage data:

We observe wages only for workers who are currently employed. But the decision to work (the selection equation) may be correlated with the wage itself: high-ability individuals are both more likely to be employed and to earn higher wages. The observed wage distribution is therefore not a random draw from the full population distribution.

If we regress $\log(\text{wage})$ on education using only employed workers:

\[E(\log(\text{wage}) \mid \text{employed}) \neq E(\log(\text{wage}) \mid \text{all workers})\]

OLS on the selected sample may overstate or understate the population return to education depending on the nature of the selection.

Formal model (Heckman two-step): The workhorse solution is Heckman's (1979) selection model, which jointly models: 1. Selection equation: $P(\text{employed} = 1 \mid z)$ — probit. 2. Outcome equation: $\log(\text{wage}) = x'\beta + \lambda(\hat{\alpha}'z) + \epsilon$ — OLS with an added inverse Mills ratio $\lambda(\cdot)$ to correct for selection.

(Advanced; covered in ECMT6002. Here, recognise the problem and understand what drives it.)

4. Truncation vs Censoring

These terms are often confused:

Concept	Definition	Example
Truncation	The entire observation (including $x$) is excluded from the sample when $y$ falls outside a range.	Study of high earners: everyone earning below $200k is entirely excluded from the dataset.
Censoring	The observation is in the dataset, but $y$ is only partially observed (recorded as a limit value).	Top-coded income: everyone earning above $200k has income recorded as "≥$200k".

In both cases, OLS on the observed data is biased because the sample no longer represents the full distribution.

Tobit model: For censored data (e.g. hours worked, which is 0 for non-workers), the Tobit estimator correctly accounts for the censoring mechanism:

\[y^* = x'\beta + u, \quad u \sim N(0, \sigma^2)$$ $$y = \max(0,\; y^*)\]

OLS on the censored sample produces estimates biased toward zero (because many observations are piled at the boundary). Tobit recovers the latent $\beta$ consistently.

5. Outliers: Leverage, Influence, and Cook's D

Outliers can severely distort OLS estimates. Two distinct concepts:

Leverage — how unusual is observation $i$'s position in $x$-space:

\[h_{ii} = x_i'(X'X)^{-1}x_i\]

High-leverage observations ($h_{ii} > 2(k+1)/n$) have the potential to distort estimates, but do not necessarily do so.

Cook's Distance — the actual influence of observation $i$ on all estimated coefficients:

\[D_i = \frac{(\hat{\beta}_{(-i)} - \hat{\beta})'(X'X)(\hat{\beta}_{(-i)} - \hat{\beta})}{(k+1)\hat{\sigma}^2}\]

Rule of thumb: $D_i > 4/n$ warrants investigation; $D_i > 1$ is often considered highly influential.

Remedies: - Inspect the flagged observation for data entry errors. - Use robust regression (e.g. M-estimation, least trimmed squares) which downweights high-influence observations. - Report results with and without the outlier to demonstrate sensitivity.

Worked Example

Setup. Suppose we use the Wooldridge mroz dataset, which contains labour market data for 753 married women, of whom only 428 are employed (report a positive wage).

Naive OLS: Regress $\log(\text{wage})$ on education, experience, and experience² using only the 428 employed women.

Problem: Women with lower wages are more likely to be out of the labour force entirely (their potential wage is below their reservation wage). The employed subsample is therefore positively selected — we see the upper portion of the wage distribution.

Direction of bias: - If low-ability/low-wage women are excluded, the observed average wage is above the population average. - The estimated return to education may be understated if higher-education women are disproportionately likely to work regardless, so there is less variation in education among the selected group. - In practice, the Heckman correction for the mroz data typically shows that the selection-corrected return to education is higher than the naive OLS estimate.

Practical takeaway: Always ask "What determines whether I observe $y$ for this unit?" If the answer involves variables correlated with $y$ itself, you have a sample selection problem.

Common Misconception

"Just drop rows with missing data and proceed — the data are probably missing at random."

This is one of the most dangerous casual assumptions in applied econometrics.

Dropping observations is only valid under MCAR, which is the strongest (and least common) assumption. In most real datasets, missingness is related to the outcome or covariates (MAR or MNAR). Simply deleting missing rows in those cases introduces selection bias that is invisible to standard diagnostic tests — your regression runs, produces plausible-looking coefficients, and you have no warning that the estimates are systematically wrong.

Always investigate the missingness pattern: 1. Tabulate missingness rates by key subgroups. 2. Regress a missingness indicator on observed covariates. 3. Compare means of $y$ and $x$ between complete and incomplete cases.

Only after this should you decide whether listwise deletion, imputation, or a selection model is appropriate.

Practice Prompts

Missing data classification. A researcher studies the effect of class size on student test scores. In 30% of schools, test scores are not reported because the schools opted out of standardised testing. Low-performing schools are more likely to opt out. (a) Which missing data mechanism (MCAR, MAR, MNAR) best describes this situation? Justify your answer. (b) What is the likely direction of bias if the researcher drops schools with missing test scores?
Truncation vs censoring. A bank studies the determinants of loan repayment amounts. Repayment is recorded as: (i) the actual dollar amount for amounts between $0 and $500,000; (ii) "$500,000+" for any repayment at or above the cap. (a) Is this truncation or censoring? Explain. (b) What model would you use to recover unbiased estimates of the effect of borrower characteristics on repayment?
Outlier diagnostics. You estimate a consumption function with $n = 200$ and $k = 4$ regressors. Observation 17 has Cook's D = 0.85 and leverage $h_{17,17} = 0.18$. (a) Is observation 17 high-leverage? (Use the $2(k+1)/n$ rule of thumb.) (b) Is it influential by Cook's D? (Use the $4/n$ rule.) (c) Outline the steps you would take before deciding whether to exclude observation 17.

Further Resources

Wooldridge, Introductory Econometrics (7e), §9.4 — Missing Data and Nonrandom Samples.
Rubin, D.B. (1976). "Inference and missing data." Biometrika, 63(3), 581–592. — Original taxonomy of MCAR/MAR/MNAR.
Heckman, J.J. (1979). "Sample selection bias as a specification error." Econometrica, 47(1), 153–161. — Nobel Prize–winning correction model.
Little, R.J.A. & Rubin, D.B. (2002). Statistical Analysis with Missing Data (2nd ed.). — Comprehensive reference.
Tobin, J. (1958). "Estimation of relationships for limited dependent variables." Econometrica, 26(1), 24–36. — Original Tobit paper.

Concept	Definition	Example
Truncation	The entire observation (including \(x\)) is excluded from the sample when \(y\) falls outside a range.	Study of high earners: everyone earning below $200k is entirely excluded from the dataset.
Censoring	The observation is in the dataset, but \(y\) is only partially observed (recorded as a limit value).	Top-coded income: everyone earning above $200k has income recorded as "≥$200k".