Lesson M04.L04: F-Tests and Joint Hypotheses
| Field | Detail |
|---|---|
| Module | Module 04 — Statistical Inference in OLS |
| Level | Graduate (ECMT5001, S1 2027) |
| Duration | 35 min |
| Learning Objective | Formulate and execute F-tests for joint hypotheses; relate the F-statistic to restricted and unrestricted SSR/R²; test the overall significance of a regression; understand why individual t-tests cannot replace joint tests |
| Provenance | Wooldridge, Introductory Econometrics, 7e, Chapter 4.5 (pp. 153–168) |
Explanation
Why We Need Joint Tests
Suppose we want to know whether education and experience are together relevant predictors of wages. We might think: just check whether both have \(|t| > 1.96\). But this is wrong.
If the t-tests for \(\hat{\beta}_1\) and \(\hat{\beta}_2\) are correlated (which they are whenever the regressors are correlated), then: - Two individually insignificant variables can be jointly significant. - Two individually significant variables can (in principle) be jointly redundant.
The correct tool for testing multiple restrictions simultaneously is the F-test.
Setting Up the Hypotheses
A joint hypothesis imposes \(q\) linear restrictions on the \(\beta\) vector. The most common case is testing that \(q\) specific coefficients are all zero:
For example, in a model with 5 regressors, we might test whether the last two are jointly irrelevant: \(H_0: \beta_4 = 0, \beta_5 = 0\) (\(q = 2\) restrictions).
Restricted vs. Unrestricted Models
The F-test compares two regressions:
Unrestricted model (UR): includes all regressors (k total); has \(SSR_{ur}\) and \(R^2_{ur}\).
Restricted model (R): imposes \(H_0\) by dropping the \(q\) restricted regressors; has \(SSR_r\) and \(R^2_r\).
Imposing restrictions always increases SSR (or at best leaves it unchanged). The question is: does the increase in SSR from imposing \(H_0\) represent more than we'd expect by chance?
The F-Statistic
where: - \(SSR_r - SSR_{ur}\) = increase in residual sum of squares from imposing restrictions - \(q\) = number of restrictions (numerator df) - \(n - k - 1\) = degrees of freedom of the unrestricted model (denominator df)
Under \(H_0\) (and CLM assumptions MLR.1–MLR.6):
An F-distribution with \(q\) and \(n-k-1\) degrees of freedom. Since larger F means larger deterioration in fit under \(H_0\), we always use a right-tailed test:
Equivalent \(R^2\) Form
Since \(SSR = SST(1 - R^2)\) and \(SST\) is the same in restricted and unrestricted models (when the restricted model still includes an intercept):
This form is convenient when software reports \(R^2\) rather than \(SSR\).
Overall Significance of the Regression
A special case: the null hypothesis that all slope coefficients are zero (none of the \(k\) regressors help explain \(y\)):
Under \(H_0\), the restricted model is just \(y_i = \beta_0 + u_i\), so \(R^2_r = 0\), and the F-statistic simplifies to:
This is the overall F-statistic reported by regression software. Rejecting this \(H_0\) means the regression has some explanatory power beyond just the mean of \(y\).
Why Individual t-Tests Are Not Sufficient for Joint Hypotheses
Suppose we test \(H_0: \beta_1 = 0\) and \(H_0: \beta_2 = 0\) individually at 5% each. With two independent tests, the probability of making at least one Type I error (false rejection) is:
For correlated tests (which is typical in regression), the size distortion can be even worse or better depending on the correlation direction. The F-test controls the joint size at exactly \(\alpha\).
Moreover, even if both t-statistics are individually below their critical values, the F-statistic can exceed its critical value if both estimators are simultaneously pushed in the same direction by the data.
Worked Example
Setup. We estimate a log-wage regression with \(n = 526\) workers and two potential regressors beyond education and gender: \(\text{exper}\) and \(\text{exper}^2\).
Unrestricted model (\(k = 4\) regressors: educ, exper, exper², female): $\(\log(\hat{\text{wage}}) = \beta_0 + \beta_1\,\text{educ} + \beta_2\,\text{exper} + \beta_3\,\text{exper}^2 + \beta_4\,\text{female} + u\)$ $\(SSR_{ur} = 97.34, \quad R^2_{ur} = 0.3625, \quad df_{ur} = 526 - 4 - 1 = 521\)$
Restricted model (\(k^* = 2\) regressors: educ, female — drops exper and exper²): $\(\log(\hat{\text{wage}}) = \beta_0 + \beta_1\,\text{educ} + \beta_4\,\text{female} + v\)$ $\(SSR_r = 103.79, \quad R^2_r = 0.3201\)$
Joint hypothesis: \(H_0: \beta_2 = 0,\, \beta_3 = 0\) (\(q = 2\) restrictions).
Step 1 — State hypotheses: $\(H_0: \beta_2 = 0 \text{ and } \beta_3 = 0 \qquad H_1: \text{at least one } \neq 0\)$
Step 2 — Compute F-statistic (SSR form): $\(F = \frac{(SSR_r - SSR_{ur})\,/\,q}{SSR_{ur}\,/\,(n-k-1)} = \frac{(103.79 - 97.34)\,/\,2}{97.34\,/\,521}\)$
Verification via R² form: $\(F = \frac{(0.3625 - 0.3201)\,/\,2}{(1 - 0.3625)\,/\,521} = \frac{0.0424\,/\,2}{0.6375\,/\,521} = \frac{0.0212}{0.001224} = 17.32\)$
(Small rounding difference; both \(\approx 17.3\).)
Step 3 — Distribution under \(H_0\): \(F \sim F_{2,\, 521}\)
Step 4 — Critical value: At \(\alpha = 1\%\), \(F_{2,\, 521, 0.01} \approx 4.62\). At \(\alpha = 5\%\), \(F_{2,\, 521, 0.05} \approx 3.00\).
Step 5 — Decision: \(17.3 \gg 4.62\). Strongly reject \(H_0\) at any conventional significance level.
Step 6 — Interpretation: Experience and experience-squared are jointly highly significant. Despite experience and \(\text{exper}^2\) being correlated (which can inflate individual SEs), together they explain substantial variation in log wages beyond education and gender.
Connecting to Individual t-Tests
For context, individual t-statistics from the unrestricted model might be: - \(t_{\text{exper}} = 0.033 / 0.008 = 4.13\) (significant at 1%) - \(t_{\text{exper}^2} = -0.0005 / 0.0001 = -5.00\) (significant at 1%)
Here individual t-tests also indicate significance, consistent with the F-test. But this won't always be the case: see the misconception section below.
Common Misconception
"Two individually insignificant t-tests mean the variables are jointly insignificant."
Consider a model where \(x_1\) and \(x_2\) are highly positively correlated. The individual standard errors of \(\hat{\beta}_1\) and \(\hat{\beta}_2\) are inflated by multicollinearity. Individually, neither may pass the threshold \(|t| > 1.96\). Yet the F-test may strongly reject \(H_0: \beta_1 = 0, \beta_2 = 0\).
Numerical illustration. Suppose:
| \(\hat{\beta}_j\) | \(se\) | \(t\) | |
|---|---|---|---|
| \(x_1\) | 1.0 | 0.8 | 1.25 |
| \(x_2\) | 0.9 | 0.8 | 1.13 |
Both t-statistics are below 1.96 — individually insignificant at 5%. Yet if \(\hat{\beta}_1\) and \(\hat{\beta}_2\) are negatively correlated (induced by positive correlation between \(x_1\) and \(x_2\)), the F-statistic for their joint significance may exceed the critical value \(F_{2, n-k-1, 0.05} \approx 3.0\).
Conversely: two individually significant coefficients could (in principle) be jointly just significant, though in practice this is less common.
Rule: Always use the F-test when testing joint hypotheses. Individual t-tests are correct only for single-coefficient tests.
Practice Prompts
Q1. A model of house prices has \(n = 148\) observations and \(k = 5\) regressors. You want to test whether the three neighbourhood characteristic variables (school quality, crime rate, pollution) are jointly significant. The unrestricted model has \(SSR_{ur} = 12{,}450\) and the restricted model (dropping those three variables) has \(SSR_r = 14{,}200\). Compute the F-statistic, state its null distribution, find the 5% critical value, and give your conclusion.
Answer
**Step 1:** $H_0: \beta_{\text{school}} = \beta_{\text{crime}} = \beta_{\text{pollution}} = 0$; $q = 3$ restrictions. **Step 2:** $$F = \frac{(14{,}200 - 12{,}450)\,/\,3}{12{,}450\,/\,(148-5-1)} = \frac{1{,}750\,/\,3}{12{,}450\,/\,142} = \frac{583.33}{87.68} = 6.66$$ **Step 3:** Under $H_0$: $F \sim F_{3,\, 142}$. **Step 4:** $F_{3,\, 142,\, 0.05} \approx 2.67$. **Step 5:** $6.66 > 2.67$. **Reject $H_0$** at 5%. The neighbourhood variables are jointly significant.Q2. In the same house price regression, the individual t-statistics are: \(t_{\text{school}} = 1.45\), \(t_{\text{crime}} = -1.50\), \(t_{\text{pollution}} = 1.60\) (all with \(df = 142\)). The two-sided 5% critical value is \(t_{142, 0.025} \approx 1.976\). All three fail individual significance tests. Is this contradictory with the F-test result in Q1? Explain.
Answer
No contradiction. This is the classic case where multicollinearity inflates individual standard errors, making each coefficient appear imprecise on its own, while the variables collectively provide substantial explanatory power. The F-test accounts for the joint distribution of the estimators — including their covariances — and correctly identifies that the three variables together significantly improve fit. Individual t-tests ignore the correlations between $\hat{\beta}_{\text{school}}$, $\hat{\beta}_{\text{crime}}$, and $\hat{\beta}_{\text{pollution}}$, and can fail to detect genuine joint significance. Conclusion: individual t-tests are not valid substitutes for the F-test when testing joint restrictions.Q3. A model has \(R^2_{ur} = 0.48\) with \(k = 6\) regressors and \(n = 200\). The restricted model (dropping the last 3 regressors) has \(R^2_r = 0.42\). Compute the F-statistic for the joint significance of those 3 variables at the 5% level.
Answer
$q = 3$, $n - k - 1 = 200 - 6 - 1 = 193$. $$F = \frac{(0.48 - 0.42)\,/\,3}{(1 - 0.48)\,/\,193} = \frac{0.06\,/\,3}{0.52\,/\,193} = \frac{0.020}{0.002694} = 7.42$$ Under $H_0$: $F \sim F_{3,\, 193}$. Critical value $F_{3,\, 193,\, 0.05} \approx 2.65$. Since $7.42 > 2.65$, **reject $H_0$**: the three variables are jointly significant at 5%.Further Resources
- Wooldridge (2019), §4.5: "Testing Multiple Linear Restrictions: The F Test" (pp. 153–168)
- Wooldridge (2019), §4.4: "Testing Hypotheses about a Single Linear Combination of the Parameters"
- Greene (2018), Econometric Analysis, 8e, §5.4: "Testing Hypotheses Using the F Distribution"
- For general linear restrictions \(R\beta = r\), see Wooldridge §4.6 or Greene §5.5