Lesson M06.L04: Weighted Least Squares
| Field | Detail |
|---|---|
| Module | 06 — Heteroskedasticity |
| Level | Graduate |
| Duration | 30–35 min |
| Learning Objective | Understand when and how to apply Weighted Least Squares (WLS); distinguish GLS from Feasible GLS; interpret WLS as a variance-stabilising transformation; recognise the risks of misspecified weights. |
| Provenance | Wooldridge Introductory Econometrics, Ch 8.4 |
Explanation
Motivation: Exploiting Known Variance Structure
Heteroskedasticity-robust standard errors (M06.L03) are the default remedy when the form of heteroskedasticity is unknown. But in some settings, economic theory or data structure gives us information about how the error variance varies across observations. In these cases, we can do better: Weighted Least Squares (WLS) exploits this structure to produce estimates that are more efficient than OLS (with or without robust SEs).
The Setup: Known Variance Structure
Assume the error variance takes the form:
where \(h_i > 0\) is a known function of the regressors (or other observable characteristics). The parameter \(\sigma^2\) is unknown but constant; only \(h_i\) varies across observations.
Examples:
- \(h_i = x_i\) (variance proportional to a regressor, e.g. income)
- \(h_i = x_i^2\) (standard deviation proportional to \(x_i\))
- \(h_i = 1/n_g\) (for group averages with group size \(n_g\))
The WLS Transformation
Define the weight \(w_i = 1/\sqrt{h_i}\). Multiply through every term in the regression by \(w_i\):
Let \(y_i^* = y_i/\sqrt{h_i}\), \(x_{ij}^* = x_{ij}/\sqrt{h_i}\), \(u_i^* = u_i/\sqrt{h_i}\).
Transformed error variance:
The transformed model is homoskedastic. Applying OLS to the transformed model is equivalent to WLS on the original model.
WLS objective function: WLS minimises the weighted sum of squared residuals:
Observations with larger variance (\(h_i\) large) receive smaller weights — they are less informative and should contribute less to estimation. Observations with smaller variance receive larger weights — they are more precise.
GLS vs Feasible GLS
Generalised Least Squares (GLS): If \(h_i\) is exactly known, WLS = GLS. By the Gauss-Markov theorem applied to the transformed model, GLS is BLUE — the Best Linear Unbiased Estimator. It dominates OLS in efficiency whenever heteroskedasticity is present.
Feasible GLS (FGLS): In practice, \(h_i\) is almost never exactly known. We estimate it from the data, typically by:
- Run OLS; save residuals \(\hat{u}_i\).
- Regress \(\log(\hat{u}_i^2)\) on the \(x\)'s (or posited determinants of variance): \(\log(\hat{u}_i^2) = \delta_0 + \delta_1 z_i + \text{error}\).
- Obtain fitted values \(\hat{g}_i\) from this auxiliary regression.
- Compute estimated weights: \(\hat{h}_i = e^{\hat{g}_i}\).
- Run WLS with weights \(1/\hat{h}_i\).
Properties of FGLS: Consistent and asymptotically more efficient than OLS. Not exactly BLUE in finite samples because \(\hat{h}_i\) is estimated. If the model for \(h_i\) is correct, FGLS gains over OLS in large samples. If the model is misspecified, FGLS can be worse (see Common Misconception).
When WLS Is Appropriate
WLS is most natural and defensible in two settings:
1. Aggregated/Group-Average Data
When each observation is an average over a group of \(n_g\) individuals, and the individual errors are homoskedastic and independent, the group-average error has variance \(\sigma^2/n_g\). The appropriate weight is \(n_g\) (weight each group by its size).
Example: County-level crime rates, where each county is an average over \(n_g\) residents. Counties with more residents provide more precise estimates of the true crime rate.
2. Known Variance from Theory or Data Structure
When the form of \(h_i\) is known from economic theory (e.g. variance proportional to income in an Engel curve), WLS with \(w_i = 1/income_i\) is the natural estimator.
Practical Guidance
- Unknown \(h_i\) (the common case): Use OLS + robust standard errors. This is simpler, more robust to misspecification, and asymptotically valid.
- Known or well-motivated \(h_i\): Use WLS for efficiency gains.
- Group data with known group sizes: Always weight by group size.
- After WLS estimation: use WLS-robust standard errors (weight-adjusted sandwich), not OLS-standard errors on the weighted model.
Worked Example
Setting: County-level data on crime rates and unemployment (\(n_c = 90\) counties).
- \(crime_c\): county crime rate (crimes per 100,000)
- \(unemp_c\): county unemployment rate (%)
- \(pop_c\): county population (thousands)
Each county observation is an average across \(pop_c\) individuals. The variance of the county-level error is inversely proportional to population: \(h_c = 1/pop_c\).
WLS model (weight each county by \(pop_c\)):
Step 1 — OLS (unweighted):
Step 2 — WLS (weighted by population):
Comparison:
| OLS | WLS | |
|---|---|---|
| \(\hat{\beta}_1\) (unemployment) | 4.7 | 5.9 |
| SE | 1.8 | 1.2 |
| \(t\)-statistic | 2.61 | 4.92 |
- WLS gives a different point estimate (5.9 vs 4.7) because high-population counties (more precise) get more weight — and if they happen to show a stronger unemployment-crime relationship, WLS upweights that.
- WLS SE is smaller (more efficient): we exploit the known variance structure.
- Both estimators are consistent, but WLS is more efficient if the variance model is correct.
Stata:
regress crime unemp [aweight = pop]
R:
lm(crime ~ unemp, data = counties, weights = pop)
Common Misconception
"WLS always gives better estimates than OLS."
Correction: WLS is more efficient than OLS only if the assumed variance function \(h_i\) is correctly specified. If you assume \(h_i = x_i\) but the true variance structure is \(h_i = x_i^2\), your weights are wrong. Misspecified WLS:
- Is still consistent (as long as the mean function is correctly specified), but
- Can be less efficient than OLS, or even yield misleading inference if the misspecification is severe.
Moreover, WLS with misspecified weights does not produce homoskedastic errors in the transformed model — so the standard WLS standard errors are also incorrect, and you would need to further apply robust SEs. This is why the practical default in applied work is OLS + robust SEs: it requires no assumption about the form of \(\sigma_i^2\) and is protected by the sandwich estimator regardless.
Practice Prompts
Prompt 1 — Weighted by group size A researcher has state-level data on average hourly wages (the average of individual wages within each state). There are \(G = 50\) states, with state sizes \(n_s\) ranging from 500,000 to 8,000,000 workers.
(a) Why is the error variance in a state-level regression likely to vary across states? (b) What weights would you apply in WLS, and why? (c) If you applied equal weights (OLS), what would be the consequence for efficiency?
Prompt 2 — FGLS procedure A researcher suspects \(\text{Var}(u_i \mid x_i) = \sigma^2 x_i^2\). Outline the FGLS procedure step by step:
(a) What is the implied weight \(w_i = 1/\sqrt{h_i}\)? (b) How would you transform the regression equation? (c) Describe the two-step estimation procedure if \(h_i\) were not known exactly but you wanted to estimate it from the residuals.
Prompt 3 — OLS vs WLS comparison In the worked example above, OLS gives \(\hat{\beta}_{unemp} = 4.7\) and WLS gives \(\hat{\beta}_{unemp} = 5.9\).
(a) Why are the two estimates different? What does the difference tell you about the data? (b) Which estimate would you report in a policy memo? Justify your answer. (c) If you discovered the population data was measured with substantial error, how would this affect your confidence in the WLS results?
Further Resources
- Wooldridge, Introductory Econometrics (7th ed.), §8.4
- Stata:
regress y x [aweight = w]for analytic weights (WLS);[pweight = w]for probability weights (survey data) - R:
lm(y ~ x, weights = w, data = d)for WLS - For FGLS with unknown form:
glm(y ~ x, family = quasi(var = "mu^2"), data = d)in R models multiplicative heteroskedasticity - Greene, Econometric Analysis (8th ed.), §9.5–9.6 — matrix treatment of GLS and FGLS
- Angrist & Pischke, Mostly Harmless Econometrics, §3.4 — when WLS is and isn't the right tool