Lesson M04.L01: Sampling Distributions of OLS Estimators
| Field | Detail |
|---|---|
| Module | Module 04 — Statistical Inference in OLS |
| Level | Graduate (ECMT5001, S1 2027) |
| Duration | 35 min |
| Learning Objective | Derive the exact and asymptotic sampling distributions of \(\hat{\beta}_j\) under MLR.1–MLR.6, and correctly identify degrees of freedom for the t-distribution |
| Provenance | Wooldridge, Introductory Econometrics, 7e, Chapter 4 (pp. 117–130) |
Explanation
MLR.1–MLR.6: The Classical Linear Model Assumptions
Recall the six assumptions of the Classical Linear Model (CLM):
| Assumption | Name | Statement |
|---|---|---|
| MLR.1 | Linearity in parameters | \(y = \beta_0 + \beta_1 x_1 + \cdots + \beta_k x_k + u\) |
| MLR.2 | Random sampling | \((x_{i1},\ldots,x_{ik}, y_i)\) are i.i.d. draws from the population |
| MLR.3 | No perfect collinearity | No exact linear relationship among regressors |
| MLR.4 | Zero conditional mean | \(E(u \mid x_1,\ldots,x_k) = 0\) |
| MLR.5 | Homoskedasticity | \(Var(u \mid x_1,\ldots,x_k) = \sigma^2\) |
| MLR.6 | Normality | \(u \mid x_1,\ldots,x_k \sim N(0, \sigma^2)\) |
MLR.1–MLR.5 give us BLUE (Gauss–Markov). MLR.6 is the additional normality assumption that pins down the exact finite-sample distribution of \(\hat{\beta}_j\).
The Exact Sampling Distribution Under MLR.1–MLR.6
Under all six CLM assumptions, the OLS estimator \(\hat{\beta}_j\) is a linear combination of the \(y_i\) values. Since \(y_i\) is itself a linear function of the normally distributed error \(u_i\) (by MLR.6), \(\hat{\beta}_j\) is also normally distributed:
where \(Var(\hat{\beta}_j) = \dfrac{\sigma^2}{SST_j(1 - R_j^2)}\), with \(SST_j = \sum_{i=1}^n (x_{ij} - \bar{x}_j)^2\) and \(R_j^2\) the \(R^2\) from regressing \(x_j\) on all other regressors.
Standardising to a \(N(0,1)\) variate (subtracting mean, dividing by SD):
This is a clean \(Z\)-statistic — but it requires knowing \(\sigma^2\), the true error variance, which is unobservable.
Replacing \(\sigma^2\) with \(\hat{\sigma}^2\): Why We Use the t-Distribution
In practice we estimate \(\sigma^2\) by:
where \(SSR\) is the sum of squared residuals, \(n\) is the number of observations, \(k\) is the number of slope regressors, and \(1\) accounts for the estimated intercept. The denominator \(n-k-1\) is the degrees of freedom (df).
Replacing \(\sigma\) with \(\hat{\sigma}\) (the standard error of the regression) gives the standard error of \(\hat{\beta}_j\):
Substituting into the standardised statistic:
This follows a t-distribution with \(n - k - 1\) degrees of freedom.
Why t, not Z? Dividing by \(\hat{\sigma}\) rather than the true \(\sigma\) introduces an extra source of randomness. More precisely:
where \(Z \sim N(0,1)\) is independent of the \(\chi^2_{n-k-1}\) term. A standard normal divided by the square root of an independent scaled chi-squared is, by definition, a t-random variable. The t-distribution has heavier tails than \(N(0,1)\), reflecting the extra uncertainty from estimating \(\sigma^2\). Ignoring this and using \(Z\) critical values in small samples understates uncertainty and causes over-rejection of true null hypotheses.
Degrees of Freedom: Where Does \(n - k - 1\) Come From?
We have \(n\) observations and are estimating \(k + 1\) parameters (\(\beta_0, \beta_1, \ldots, \beta_k\)). The residuals \(\hat{u}_i\) must satisfy \(k + 1\) constraints (the OLS first-order conditions), so there are only \(n - (k+1) = n - k - 1\) "free" pieces of information left to estimate \(\sigma^2\).
| Model | Parameters estimated | Degrees of freedom |
|---|---|---|
| Simple regression (\(k=1\)) | 2 (\(\beta_0, \beta_1\)) | \(n-2\) |
| Multiple regression (\(k\) regressors) | \(k+1\) | \(n-k-1\) |
Large-Sample (Asymptotic) Justification: Dropping MLR.6
MLR.6 (normality of \(u\)) is a strong assumption. In large samples, the CLT provides an asymptotic justification:
- Under MLR.1–MLR.5 only (no normality), \(\hat{\beta}_j\) is still consistent and asymptotically normal.
- As \(n \to \infty\): \(t_{n-k-1} \to N(0,1)\) (the t and Z distributions converge).
- For large \(n\), the t-statistics and p-values reported by regression software remain valid even without MLR.6.
- Practical rule: for \(n \geq 120\) or so, the difference between \(t\) and \(Z\) critical values is negligible (e.g. \(t_{120, 0.025} \approx 1.980\) vs. \(z_{0.025} = 1.960\)). For \(n = 20\), \(t_{18, 0.025} = 2.101\) — meaningfully larger.
Worked Example
Setup. You estimate the following model using \(n = 52\) monthly observations:
Step 1 — Identify parameters. Number of slope regressors: \(k = 3\). Number of parameters including intercept: \(k + 1 = 4\).
Step 2 — Compute degrees of freedom. $\(df = n - k - 1 = 52 - 3 - 1 = 48\)$
Step 3 — Find the t critical value for a two-sided 5% test.
We need \(t_{48,\, 0.025}\) (upper 2.5% tail). From t-tables with 48 df, this is approximately \(2.011\).
(For comparison: \(z_{0.025} = 1.960\). Using Z instead would give a critical value 0.05 units smaller — underestimating uncertainty slightly.)
Step 4 — State the sampling distribution of any \(\hat{\beta}_j\).
Under MLR.1–MLR.6: $\(\frac{\hat{\beta}_j - \beta_j}{se(\hat{\beta}_j)} \sim t_{48}\)$
Conclusion. With 52 observations and 3 regressors, our t-statistics follow a \(t_{48}\) distribution. A two-sided test at 5% significance rejects when \(|t| > 2.011\).
Common Misconception
"With a large sample, I can just use z = 1.96 instead of the t-critical value."
This is approximately (not exactly) correct, and only for large \(n\). The t-distribution is exact under MLR.1–MLR.6 regardless of sample size. Using \(z = 1.96\) for a sample of \(n = 25\) with \(k = 3\) regressors means using a critical value of 1.96 when the correct value is \(t_{21, 0.025} = 2.080\). At \(n = 25\), the z-based test will reject \(H_0\) too often — it has size greater than 5%.
Rule: Always use \(t_{n-k-1}\) critical values. With large \(n\) (say \(>120\)), \(t\) and \(z\) are nearly identical and the distinction is immaterial.
Practice Prompts
Q1. A researcher estimates a wage regression with \(n = 526\) workers and \(k = 5\) regressors (education, experience, experience², female, urban). What is the degrees of freedom for the t-distribution used in inference? What is the 5% two-sided t critical value? How does it compare to \(z_{0.025} = 1.960\)?
Answer
$df = n - k - 1 = 526 - 5 - 1 = 520$. From t-tables (or note that for df $\geq 120$, the t-distribution is very close to normal): $t_{520, 0.025} \approx 1.964$. This is virtually identical to $z_{0.025} = 1.960$. With $n = 526$, using $z$ instead of $t$ is inconsequential.Q2. A student estimates a model with \(n = 30\) and \(k = 4\) regressors, and quotes a t-statistic of 2.02 as significant at 5% (two-sided) using the \(z\) critical value of 1.96. Is this conclusion correct?
Answer
No. With $n = 30$ and $k = 4$: $df = 30 - 4 - 1 = 25$. The correct critical value is $t_{25, 0.025} = 2.060$. Since $|t| = 2.02 < 2.060$, we **fail to reject** $H_0$ at the 5% level using the correct t critical value. The student's conclusion is wrong — using $z$ leads to over-rejection in small samples.Q3. Explain why \(\frac{\hat{\beta}_j - \beta_j}{se(\hat{\beta}_j)}\) follows a t-distribution rather than a normal distribution, and state precisely what changes this distribution as the sample grows.
Answer
The statistic is the ratio of a standard normal ($\frac{\hat{\beta}_j - \beta_j}{\sigma \cdot c_j}$ where $c_j$ is a known constant) to the square root of an independent scaled chi-squared random variable ($\hat{\sigma}^2 / \sigma^2 \sim \chi^2_{n-k-1} / (n-k-1)$). This ratio defines a $t_{n-k-1}$ random variable by construction. As $n \to \infty$, the chi-squared term concentrates around its expectation of 1 (by the LLN), so the denominator converges in probability to 1, and the ratio converges in distribution to $N(0,1)$. Formally: $t_{n-k-1} \xrightarrow{d} N(0,1)$ as $n-k-1 \to \infty$.Further Resources
- Wooldridge (2019), Introductory Econometrics, 7e, §4.1 (pp. 117–124): "Sampling Distributions of the OLS Estimators"
- Wooldridge Appendix C.5: Chi-square, t, and F distributions
- Davidson & MacKinnon (2004), Econometric Theory and Methods, §4.2
- For asymptotic theory: White (2001), Asymptotic Theory for Econometricians, Ch. 2