Lesson M07.L01: Specification Errors and Model Selection
| Field | Detail |
|---|---|
| Module | 07 — Specification Issues |
| Level | Graduate (ECMT5001 S1 2027) |
| Duration | 30–35 min |
| Learning Objective | Understand functional form misspecification, apply the RESET test, and use information criteria to compare non-nested models; distinguish the consequences of under- versus over-specification. |
| Provenance | Wooldridge, Introductory Econometrics, Ch 9.1 |
Explanation
1. Functional Form Misspecification
A model is functionally misspecified when the assumed relationship between regressors and the dependent variable does not match the true data-generating process. The most common version: imposing linearity when the true relationship is nonlinear.
Suppose the true model is:
but you estimate only:
The omitted \(x^2\) term is correlated with \(x\), so by the OVB formula (recall M03.L04), \(\hat{\beta}_1\) is biased and inconsistent. Functional form misspecification is therefore a special case of omitted variable bias where the omitted variable is a transformation of an included regressor.
Other common functional form errors: - Using a linear model when a log–linear (or log–log) form is more appropriate. - Omitting interaction terms when the marginal effect of one variable depends on another. - Fitting a straight line through a relationship that curves back.
2. The RESET Test (Ramsey Regression Equation Specification Error Test)
The RESET test provides a general diagnostic for functional form misspecification. The idea: if the model is correctly specified, adding powers of the fitted values \(\hat{y}\) to the regression should not significantly improve fit—because \(\hat{y}\) is already a linear combination of the regressors.
Procedure:
Step 1. Estimate the original (restricted) model by OLS:
Obtain fitted values \(\hat{y}_i\).
Step 2. Construct an augmented (unrestricted) model:
Step 3. Test the null hypothesis:
using an F-test (joint significance of the added powers). The test statistic:
follows an \(F_{2,\, n-k-3}\) distribution under \(H_0\).
Step 4. Interpret the outcome: - Reject \(H_0\) → evidence of functional form misspecification; consider transformations, interactions, or polynomial terms. - Fail to reject \(H_0\) → no evidence detected by this test (but see Common Misconception).
Why \(\hat{y}^2\) and \(\hat{y}^3\)? These are smooth nonlinear functions of all included regressors. If any nonlinear combination of the \(x\)'s belongs in the model, it will likely be correlated with \(\hat{y}^2\) or \(\hat{y}^3\).
3. Non-Nested Model Selection: Adjusted R², AIC, BIC
When two models are non-nested (neither is a restricted version of the other), standard F-tests do not apply. Example:
- Model A: \(\log(\text{wage}) = \beta_0 + \beta_1 \text{educ} + \beta_2 \text{exper} + u\)
- Model B: \(\log(\text{wage}) = \beta_0 + \beta_1 \text{educ} + \beta_2 \text{tenure} + u\)
Use information criteria to compare:
| Criterion | Formula | Preference |
|---|---|---|
| Adjusted R² | \(\bar{R}^2 = 1 - \frac{SSR/(n-k-1)}{SST/(n-1)}\) | Higher is better |
| AIC | \(\ln(\hat{\sigma}^2) + \frac{2(k+1)}{n}\) | Lower is better |
| BIC | \(\ln(\hat{\sigma}^2) + \frac{(k+1)\ln n}{n}\) | Lower is better |
BIC vs AIC: BIC penalises complexity more heavily (via \(\ln n\) vs \(2\)), so it tends to select more parsimonious models, especially for large samples.
Caution: Information criteria compare models on the same sample and same dependent variable. You cannot use R² (or AIC/BIC in the standard form) to compare a model with \(y\) as dependent variable against one with \(\log(y)\)—different scales.
4. Under-Specification (Omitted Variable Bias)
Covered in depth in M03.L04. Brief recap:
If the true model is \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + u\) but you omit \(x_2\):
where \(\tilde{\delta}_1\) is the coefficient from regressing \(x_2\) on \(x_1\). Unless \(\beta_2 = 0\) or \(x_1 \perp x_2\), OLS is biased and inconsistent — the most serious specification error.
5. Over-Specification (Irrelevant Variables)
Suppose the true model has \(k\) regressors but you include an irrelevant variable \(x_{k+1}\) (whose true coefficient \(\beta_{k+1} = 0\)). OLS is:
- Still unbiased and consistent for all coefficients (including \(\hat{\beta}_{k+1} \approx 0\) in expectation).
- Less efficient: including an irrelevant variable inflates the variance of \(\hat{\beta}_j\) for the other regressors (via multicollinearity), producing wider confidence intervals.
There is thus an asymmetry in the cost of specification errors:
| Error type | Bias? | Efficiency? |
|---|---|---|
| Omit a relevant variable (correlated with \(x\)) | Yes — biased, inconsistent | — |
| Include an irrelevant variable | No — still unbiased | Worse — higher variance |
The practical implication: omitting a relevant variable is almost always worse. But over-specification is not free—if you include many irrelevant controls you may lose the precision needed to detect true effects.
Worked Example
Setup. Consider a wage equation estimated on \(n = 526\) observations from Wooldridge's wage1 dataset:
RESET Test Application.
- Save \(\hat{y}_i = \widehat{\log(\text{wage})}_i\) from OLS.
- Add \(\hat{y}^2\) and \(\hat{y}^3\) as additional regressors.
- Run F-test for their joint significance.
Suppose the resulting F-statistic is \(F = 4.67\) with \(p\text{-value} = 0.010\).
Interpretation.
At the 5% level, we reject \(H_0\): there is evidence of functional form misspecification in this specification. The significant F-statistic suggests a nonlinear function of the regressors (or an omitted interaction) is correlated with \(\hat{y}^2\) or \(\hat{y}^3\).
Follow-up actions: - Test whether \(\text{exper}^2\) or \(\text{tenure}^2\) should be added. - Consider whether interactions (e.g. educ × tenure) improve fit. - Re-run RESET after augmenting the model; if \(p > 0.10\), the new specification passes.
Common Misconception
"Passing the RESET test means the model is correctly specified."
This is false. The RESET test is a necessary check, not a sufficient one. It has power only against misspecifications that cause \(\hat{y}^2\) or \(\hat{y}^3\) to be significant — a particular class of functional form errors. It will not detect:
- Omitted variables that are uncorrelated with \(\hat{y}^2, \hat{y}^3\) (e.g. a purely additive shift from an omitted dummy).
- Heteroskedasticity.
- Endogeneity or reverse causality.
- Wrong choice of sample or incorrect coding.
Think of RESET as a smoke alarm: silence means no smoke detected, not that there's no fire.
Practice Prompts
-
RESET mechanics. You estimate \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + u\) with \(n = 200\). The restricted SSR is 450 and the unrestricted SSR (after adding \(\hat{y}^2, \hat{y}^3\)) is 420. Compute the RESET F-statistic and state your conclusion at the 5% significance level. (Use \(F_{2,195}\) critical value ≈ 3.04.)
-
Non-nested model comparison. Model A (\(k = 3\)): \(\hat{\sigma}^2_A = 0.85\), \(n = 150\). Model B (\(k = 5\)): \(\hat{\sigma}^2_B = 0.80\), \(n = 150\). Compute AIC and BIC for each model. Which does AIC prefer? Which does BIC prefer? Explain why they may differ.
-
Under- vs over-specification. A researcher estimates a salary model. True model: \(\text{salary} = \beta_0 + \beta_1 \text{educ} + \beta_2 \text{ability} + u\). She cannot observe ability. (a) What happens to \(\hat{\beta}_1\) if ability is positively correlated with educ and positively affects salary? (b) Separately, she adds an irrelevant variable (respondent's shoe size). What happens to \(\hat{\beta}_1\) now? Is it biased? What changes?
Further Resources
- Wooldridge, Introductory Econometrics (7e), §9.1 — Functional Form Misspecification; RESET test.
- Ramsey, J.B. (1969). "Tests for specification errors in classical linear least squares regression analysis." Journal of the Royal Statistical Society B, 31(2), 350–371.
- Davidson, R. & MacKinnon, J.G. (1993). Estimation and Inference in Econometrics, Ch 3 — model specification and non-nested tests.
- See also: M03.L04 (Omitted Variable Bias) for the formal OVB derivation.