Lesson M03.L01: The Multiple Regression Model

Module: M03 — Multiple Regression Analysis
Level: Graduate (ECMT5001, USYD S1 2027)
Duration: 35 min
Learning Objective: Specify and interpret a multiple regression model; explain why controlling for additional regressors produces ceteris paribus partial effects; recognise the link between simple-regression coefficient bias and omitted variables.
Provenance: Wooldridge, Introductory Econometrics, Chapter 3 (Sections 3.1–3.2)

Explanation

Why Multiple Regression?

Simple regression estimated the relationship between $y$ and a single explanatory variable. That approach is rarely sufficient in economics because the world is multivariate. Any factor that

affects $y$, and
correlates with the included regressor,

will distort the estimate if left out — a problem we formalise in M03.L04 as omitted variable bias (OVB).

Multiple regression solves this by including the relevant controls directly in the model, thereby isolating the partial or ceteris paribus effect of each regressor.

The General Model

\[y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_k x_k + u\]

Symbol	Meaning
$y$	Dependent (outcome) variable
$x_1, \ldots, x_k$	Independent (explanatory) variables — the regressors
$\beta_0$	Intercept — value of $y$ when all $x_j = 0$
$\beta_j$ ($j \geq 1$)	Partial effect of $x_j$ on $y$, all else equal
$u$	Error term — everything that affects $y$ but is not in the model

The model is called linear in parameters because each $\beta_j$ enters linearly. The $x_j$ themselves can be nonlinear transformations (logs, squares, interactions) — that is fine.

Interpretation of $\beta_j$: Ceteris Paribus

\[\frac{\partial y}{\partial x_j} = \beta_j \quad (\text{holding } x_1, \ldots, x_{j-1}, x_{j+1}, \ldots, x_k \text{ fixed})\]

Reading: A one-unit increase in $x_j$ is associated with a $\beta_j$-unit change in $y$, holding all other regressors constant.

This "holding constant" interpretation is the most important conceptual gain over simple regression. In the real world we cannot run controlled experiments for most economic questions, but multiple regression gives us the statistical equivalent of holding other factors fixed.

Wage Equation Example

Consider estimating returns to education in the Australian labour market:

\[\text{wage} = \beta_0 + \beta_1 \,\text{educ} + \beta_2 \,\text{exper} + \beta_3 \,\text{tenure} + u\]

where:

Variable	Units	Meaning
`wage`	AUD per hour	Hourly wage
`educ`	years	Years of formal education
`exper`	years	Years of general work experience
`tenure`	years	Years with current employer

$\beta_1$ = return to one additional year of education, holding experience and tenure fixed
$\beta_2$ = return to one additional year of experience, holding education and tenure fixed
$\beta_3$ = return to one additional year of tenure, holding education and experience fixed

Without controlling for experience and tenure, a simple regression of wage on education would conflate the returns to education with the fact that more-educated workers tend to accumulate different experience profiles.

How Adding Variables Changes Coefficients

Consider the simple regression:

\[\widetilde{\text{wage}} = \tilde{\beta}_0 + \tilde{\beta}_1 \,\text{educ}\]

versus the multiple regression above. In general $\tilde{\beta}_1 \neq \hat{\beta}_1$ because:

exper and tenure both affect wage ($\beta_2, \beta_3 \neq 0$)
exper and tenure correlate with educ

The simple-regression coefficient $\tilde{\beta}_1$ picks up direct returns to education plus the indirect correlation through experience/tenure. The multiple regression coefficient $\hat{\beta}_1$ isolates only the direct effect. The difference is OVB (formalised in M03.L04).

Key insight: The multiple regression coefficient on $x_1$ is numerically equivalent to what you would get by (1) regressing $x_1$ on all other regressors and taking the residuals $\tilde{r}_1$, then (2) regressing $y$ on $\tilde{r}_1$ alone. This is the Frisch–Waugh–Lovell theorem and makes the "partialling out" interpretation precise.

When Adding Variables Hurts

A common misconception is that you should always add more regressors. This is wrong for two reasons:

Inflated standard errors: Including irrelevant variables (variables with $\beta_j = 0$) does not reduce bias (there is nothing to remove) but does inflate the variances of all estimators — you lose efficiency. (Formalised in M03.L05.)
Multicollinearity: Highly correlated regressors make it hard for OLS to tell their effects apart. Standard errors balloon, and coefficients become imprecise even if individually significant in the population.
Overfitting: With too many regressors relative to sample size, in-sample fit rises spuriously while out-of-sample forecasts worsen.

The discipline of deciding which variables belong in the regression (i.e., have nonzero $\beta_j$ and are not redundant) is specification analysis — a central econometric skill.

Worked Example

Setup: A researcher estimates the following model using 526 Australian workers from the HILDA survey:

\[\widehat{\ln(\text{wage})} = 0.284 + 0.092\,\text{educ} + 0.041\,\text{exper} - 0.00071\,\text{exper}^2 + 0.022\,\text{tenure}\]

(a) Interpret $\hat{\beta}_1 = 0.092$.

Because the dependent variable is $\ln(\text{wage})$, a one-year increase in education is associated with approximately a $0.092 \times 100 = 9.2\%$ increase in hourly wage, holding experience and tenure constant.

(b) Why is $\text{exper}^2$ included?

Labour economists typically find that wage growth from experience is concave — each additional year adds less as the worker ages. Including $\text{exper}^2$ allows for this diminishing return. The marginal effect of experience on $\ln(\text{wage})$ is:

\[\frac{\partial \ln(\text{wage})}{\partial \text{exper}} = 0.041 - 2(0.00071)\,\text{exper} = 0.041 - 0.00142\,\text{exper}\]

At $\text{exper} = 10$ years this equals $0.041 - 0.0142 = 0.0268$, i.e., approximately 2.7% per year.

(c) Why does the intercept (0.284) have little economic meaning here?

$\beta_0$ is the predicted $\ln(\text{wage})$ when educ = exper = tenure = 0 — a worker with no education or work history. This is outside the support of the data and has no economic interpretation. Do not over-read intercepts in practice.

Common Misconception

Adding more variables always improves the regression

False. Adding an irrelevant variable ($\beta_j = 0$ in the population) does not bias the remaining coefficients but inflates their standard errors, making hypothesis tests less powerful. The adjusted $R^2$ (covered in M03.L02) can actually fall when an irrelevant regressor is added. Use economic theory and prior evidence to justify each variable's inclusion — do not kitchen-sink.

Practice Prompts

Model specification. A researcher wants to study the effect of class size on student test scores (NAPLAN) in Australian primary schools. List three variables you would add as controls. For each, explain (a) why it likely affects test scores and (b) why it might correlate with class size.
Ceteris paribus. Suppose you estimate: $$\widehat{\text{crime rate}} = 15.3 - 0.42\,\text{police officers} + 0.61\,\text{unemployment rate}$$ A city council argues: "Cities with more police have higher crime rates, so hiring police causes crime." Critique this reasoning using the concept of ceteris paribus and explain what the regression actually tells us.
Effect of adding a variable. You first estimate $\tilde{\beta}_1 = 0.12$ (simple regression of log-wage on education). After adding experience and tenure you find $\hat{\beta}_1 = 0.09$. What does this change suggest about the correlation between education and the omitted variables? Is the bias upward or downward in the simple regression?

Further Resources

Wooldridge, Ch 3.1–3.2 (pp. 68–85)
Angrist & Pischke, Mostly Harmless Econometrics, Ch 3 (Frisch–Waugh interpretation)
HILDA Survey data dictionary (University of Melbourne) — for Australian wage examples

Symbol	Meaning
\(y\)	Dependent (outcome) variable
\(x_1, \ldots, x_k\)	Independent (explanatory) variables — the regressors
\(\beta_0\)	Intercept — value of \(y\) when all \(x_j = 0\)
\(\beta_j\) (\(j \geq 1\))	Partial effect of \(x_j\) on \(y\), all else equal
\(u\)	Error term — everything that affects \(y\) but is not in the model