Lesson M01.L05: Sampling Distributions and Sample Statistics
| Field | Detail |
|---|---|
| Module | Module 01 — Probability & Statistics Review |
| Level | Graduate (ECMT5001, USYD) |
| Duration | 30–35 minutes |
| Learning Objective | Define and distinguish population vs sample; understand the sampling distribution of the sample mean; apply the Central Limit Theorem; compute standard errors; articulate why the CLT justifies large-sample inference in econometrics |
| Provenance | Wooldridge Introductory Econometrics, Appendix C.1–C.3 |
Explanation
Population vs Sample
In econometrics we almost always want to know something about a population — the complete set of units (individuals, firms, countries) to which our conclusions are meant to apply. Because populations are typically unobservable or too large to measure fully, we work with a sample — a subset drawn from the population.
| Concept | Population | Sample |
|---|---|---|
| Size | \(N\) (often unknown/infinite) | \(n\) (known, finite) |
| Mean | \(\mu\) (parameter, unknown) | \(\bar{X}\) (statistic, computed) |
| Variance | \(\sigma^2\) (parameter, unknown) | \(s^2\) (statistic, computed) |
| Correlation | \(\rho\) | \(r\) |
Random sampling is the process of drawing observations such that each unit has a known probability of selection, and draws are independent. The standard econometric framework assumes simple random sampling (SRS): each unit is equally likely to be selected, and draws are i.i.d. (independent and identically distributed).
Sample Mean as an Estimator
Given a random sample \(\{X_1, X_2, \ldots, X_n\}\) where each \(X_i \overset{\text{i.i.d.}}{\sim} (\mu, \sigma^2)\), the sample mean is:
The sample mean is an estimator of the population mean \(\mu\). It is a random variable — its value changes from sample to sample.
Properties of Estimators
An estimator \(\hat{\theta}\) of a parameter \(\theta\) is evaluated on three criteria:
1. Unbiasedness
An estimator is unbiased if its expected value (averaging over all possible samples) equals the true parameter. The sample mean \(\bar{X}\) is unbiased for \(\mu\):
The sample variance \(s^2 = \frac{1}{n-1}\sum(X_i - \bar{X})^2\) is unbiased for \(\sigma^2\) (the \(n-1\) denominator corrects for the degrees of freedom used to estimate \(\bar{X}\)).
2. Efficiency (Minimum Variance)
Among all unbiased estimators of \(\theta\), the efficient estimator has the smallest variance. A more efficient estimator wastes less information from the data. The Cramér–Rao lower bound gives the theoretical minimum variance for an unbiased estimator.
3. Consistency
An estimator \(\hat{\theta}_n\) is consistent if it converges in probability to the true value as \(n \to \infty\):
The sample mean is consistent for \(\mu\) by the Law of Large Numbers (LLN): as the sample grows, \(\bar{X}\) gets arbitrarily close to \(\mu\).
Mnemonic for graduate study: Unbiasedness is a finite-sample property (holds for any \(n\)). Consistency is an asymptotic property (holds as \(n \to \infty\)). An estimator can be biased but consistent (e.g., MLE of \(\sigma^2\) divides by \(n\) but still converges to \(\sigma^2\)), or unbiased but inconsistent (theoretically possible, though rare in practice).
Sampling Distribution of \(\bar{X}\)
Because \(\bar{X}\) is a random variable, it has its own distribution — the sampling distribution. Its mean and variance follow directly from the properties of \(X_i\):
(The cross terms vanish because the \(X_i\) are independent.)
Standard error of the mean:
In practice \(\sigma\) is unknown, so we replace it with the sample standard deviation \(s\):
Key observation: \(SE(\bar{X})\) shrinks at rate \(1/\sqrt{n}\). Quadrupling the sample size halves the standard error.
Central Limit Theorem (CLT)
The Central Limit Theorem is arguably the most important result in applied statistics:
Theorem: Let \(X_1, \ldots, X_n\) be i.i.d. with mean \(\mu\) and finite variance \(\sigma^2 > 0\). Then as \(n \to \infty\):
\[\bar{X} \xrightarrow{d} N\!\left(\mu,\ \frac{\sigma^2}{n}\right)\]Equivalently, the standardised sample mean converges in distribution to a standard normal:
\[Z_n = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} N(0, 1)\]
What the CLT says: No matter what shape the underlying population distribution has (uniform, exponential, skewed, multimodal — as long as it has finite variance), the distribution of the sample mean becomes approximately normal as the sample size grows.
What the CLT does NOT say: The individual observations \(X_i\) are not normally distributed. Only the sample mean \(\bar{X}\) is approximately normal. Many students confuse these — see the Common Misconception section.
In practice: The approximation is generally good for \(n \geq 30\) with symmetric populations, and may require \(n \geq 100\)–\(200\) for heavily skewed distributions.
Why the CLT Matters for Econometrics
The CLT provides the large-sample (asymptotic) justification for the t-tests and F-tests used throughout econometrics:
-
OLS estimators \(\hat{\beta}_0, \hat{\beta}_1\) can be written as weighted sums of the \(Y_i\) (hence weighted sums of i.i.d. observations). By the CLT, their sampling distributions are approximately normal in large samples.
-
t-tests for regression coefficients: the test statistic \(t = \hat{\beta}_1 / SE(\hat{\beta}_1)\) follows (approximately) a \(t\)-distribution in finite samples and converges to \(N(0,1)\) as \(n \to \infty\).
-
Without the CLT, we would need to assume the error term \(u\) is normally distributed (Wooldridge's Assumption MLR.6) to justify exact finite-sample t and F tests. With the CLT, normality of \(u\) is only needed for small samples — large samples let the CLT do the work.
Worked Example
Scenario: A USYD researcher samples hourly wages (in AUD) from 64 casual workers. The sample yields \(\bar{X} = \$32.50\) and \(s = \$8.00\).
Step 1: Compute the standard error.
Step 2: Apply the CLT.
Since \(n = 64 \geq 30\) and wages are roughly symmetric, the CLT guarantees:
Step 3: Compute a probability.
What is the probability that a sample mean from repeated samples of size 64 would be at least \(\$34.00\)?
So there is about a 6.7% chance of observing a sample mean this high or higher if the true population mean is \(\$32.50\).
Step 4: Construct a 95% confidence interval for \(\mu\).
We are 95% confident the population mean hourly wage lies between $30.54 and $34.46.
Interpretation: The CLT transforms a point estimate ($32.50) into an interval that quantifies our uncertainty about the population mean, even though wages are not normally distributed in the population.
Common Misconception
"The CLT says that the data is approximately normally distributed in large samples."
This is false. The CLT says that the sampling distribution of the sample mean \(\bar{X}\) is approximately normal — not the individual observations.
Illustration: Suppose individual wages follow a highly right-skewed distribution (many low-wage workers, a few very high earners). Even with \(n = 1000\):
- A histogram of the raw wage data will still look right-skewed.
- A histogram of sample means (each computed from 1000 draws, repeated across many simulated datasets) will look approximately normal.
These are completely different objects. The CLT applies to the second, not the first.
Why this matters: - For inference: t-tests on \(\bar{X}\) are valid in large samples even if individual wages are skewed (CLT justifies normality of \(\bar{X}\)). - For modelling: If you assume individual wages are normal for some modelling purpose (e.g., MLE), you need to verify this separately — the CLT does not help you there.
Practice Prompts
Prompt 1
Weekly household incomes in a city have population mean \(\mu = \$2{,}500\) and population standard deviation \(\sigma = \$600\). A random sample of \(n = 100\) households is drawn.
(a) What is \(E(\bar{X})\) and \(SE(\bar{X})\)? (b) Using the CLT, what is the probability that the sample mean exceeds \(\$2{,}600\)? (c) What sample size would be needed to halve the standard error?
Answer
**(a)** $$E(\bar{X}) = \mu = \$2{,}500$$ $$SE(\bar{X}) = \frac{\sigma}{\sqrt{n}} = \frac{600}{\sqrt{100}} = \frac{600}{10} = \$60$$ **(b)** By CLT, $\bar{X} \approx N(2500, 60^2)$. $$P(\bar{X} > 2600) = P\!\left(Z > \frac{2600 - 2500}{60}\right) = P(Z > 1.667) \approx 0.0478$$ About a 4.8% chance. **(c)** We need $SE(\bar{X}) = 60/2 = 30$, so: $$\frac{600}{\sqrt{n}} = 30 \implies \sqrt{n} = 20 \implies n = 400$$ Quadrupling the sample size halves the standard error (the $1/\sqrt{n}$ relationship).Prompt 2
An economist estimates a sample variance \(s^2 = 25\) from \(n = 36\) observations.
(a) Compute the estimated standard error of the mean. (b) Explain in one sentence why the denominator of \(s^2 = \frac{1}{n-1}\sum(X_i - \bar{X})^2\) uses \(n-1\) instead of \(n\). (c) Suppose a colleague uses \(\tilde{s}^2 = \frac{1}{n}\sum(X_i - \bar{X})^2\) instead. Is \(\tilde{s}^2\) unbiased for \(\sigma^2\)? Is it consistent?
Answer
**(a)** $$\widehat{SE}(\bar{X}) = \frac{s}{\sqrt{n}} = \frac{\sqrt{25}}{\sqrt{36}} = \frac{5}{6} \approx 0.833$$ **(b)** Using $n-1$ (Bessel's correction) corrects for the fact that $\bar{X}$ was estimated from the same data — the deviations $X_i - \bar{X}$ sum to zero by construction, so only $n-1$ of them are "free", leaving one degree of freedom consumed. **(c)** - $\tilde{s}^2$ is **biased**: $E(\tilde{s}^2) = \frac{n-1}{n}\sigma^2 \neq \sigma^2$. - $\tilde{s}^2$ is **consistent**: as $n \to \infty$, $\frac{n-1}{n} \to 1$, so $\tilde{s}^2 \xrightarrow{p} \sigma^2$. This is an example of a biased but consistent estimator.Prompt 3
Three different estimators of the population mean \(\mu\) are proposed from a sample of size \(n\):
(a) Is each estimator unbiased for \(\mu\)? (Assume i.i.d. data with mean \(\mu\) and variance \(\sigma^2\).) (b) Compute \(\text{Var}(\hat{\mu}_1)\), \(\text{Var}(\hat{\mu}_2)\), \(\text{Var}(\hat{\mu}_3)\) for general \(n\). (c) Which estimator is most efficient for \(n > 2\)? Which is least efficient? (d) Is \(\hat{\mu}_2\) consistent?
Answer
**(a)** All three are unbiased: - $E(\bar{X}) = \mu$ ✓ - $E(X_1) = \mu$ ✓ - $E\!\left(\frac{X_1+X_2}{2}\right) = \frac{\mu + \mu}{2} = \mu$ ✓ **(b)** - $\text{Var}(\hat{\mu}_1) = \sigma^2/n$ - $\text{Var}(\hat{\mu}_2) = \sigma^2$ - $\text{Var}(\hat{\mu}_3) = \frac{1}{4}(\sigma^2 + \sigma^2) = \sigma^2/2$ **(c)** For $n > 2$: $\hat{\mu}_1$ is most efficient (smallest variance $\sigma^2/n$); $\hat{\mu}_2$ is least efficient (variance $\sigma^2$, independent of $n$). **(d)** $\hat{\mu}_2$ is **not consistent**: $\text{Var}(\hat{\mu}_2) = \sigma^2$ for all $n$ — adding more data never reduces the variance of an estimator that ignores all data except the first observation.Further Resources
- Wooldridge — Introductory Econometrics: A Modern Approach, Appendix C.1–C.3 (estimation, sampling properties)
- 3Blue1Brown — But what is the Central Limit Theorem? — outstanding visual intuition for why the CLT works
- USYD ECMT1020 Notes — Section 1 (sample mean, variance formulas) for USYD-specific notation on \(s^2\) and \(\bar{X}\)