Lesson M01.L03: Random Variables & Probability Distributions
| Field | Value |
|---|---|
| Module | M01 — Probability & Statistics Review |
| Level | Graduate (ECMT5001) |
| Duration | ~75 minutes |
| Learning Objective | Characterise random variables by their distributions, expected values, and variances; standardise normal random variables; and identify when the \(t\), \(\chi^2\), and \(F\) distributions arise in econometric testing. |
| Provenance | Wooldridge Appendices B & C; ECMT1020 Weeks 1–3 (R.2–R.4, R.8–R.9) |
Explanation
1. Random Variables
A random variable (RV) \(X\) is a function that assigns a real number to each outcome in a sample space. We distinguish two types:
| Discrete | Continuous | |
|---|---|---|
| Values | Countable set \(\{x_1, x_2, \ldots\}\) | Uncountable interval (e.g., \(\mathbb{R}\)) |
| Distribution | Probability mass function (PMF) | Probability density function (PDF) |
| Probability of exact value | \(P(X = x_i) > 0\) possible | \(P(X = x) = 0\) always |
| Economic example | Number of workers laid off | Wage rate (AUD/hr) |
2. Probability Mass Function (Discrete)
The PMF \(p(x_i) = P(X = x_i)\) satisfies:
Example (Bernoulli): A firm wins a government contract with probability \(\theta = 0.3\).
3. Probability Density Function (Continuous)
For continuous \(X\), the PDF \(f(x)\) gives the density of probability (not the probability itself):
Probabilities are areas under the curve:
Note: \(P(X = c) = 0\) for any single point \(c\) — this is not a contradiction; it means exact values have zero measure.
4. Cumulative Distribution Function (CDF)
The CDF applies to both discrete and continuous RVs:
Key properties: - \(F\) is non-decreasing. - \(F(-\infty) = 0\); \(F(+\infty) = 1\). - For continuous \(X\): \(P(a < X \leq b) = F(b) - F(a)\). - For continuous \(X\): \(f(x) = F'(x)\) (the PDF is the derivative of the CDF).
5. Expected Value
The expected value (population mean) \(\mu = E(X)\) is the probability-weighted average of all possible values.
Discrete: $\(E(X) = \sum_{i} x_i\, p(x_i)\)$
Continuous: $\(E(X) = \int_{-\infty}^{+\infty} x\, f(x)\, dx\)$
Both are weighted averages — the weights are probabilities (discrete) or probability densities (continuous).
Expectation Rules
For constants \(a\), \(b\) and random variables \(X\), \(Y\):
| Rule | Formula |
|---|---|
| Linearity | \(E(aX + b) = a\,E(X) + b\) |
| Additivity | \(E(X + Y) = E(X) + E(Y)\) |
| Constant | \(E(b) = b\) |
| Independence | \(E(XY) = E(X)\,E(Y)\) if \(X \perp Y\) |
6. Variance
The variance \(\text{Var}(X) = \sigma^2\) measures the average squared deviation from the mean:
Computational shortcut (expand and simplify using linearity of \(E\)):
Variance Rules
| Rule | Formula | Intuition |
|---|---|---|
| Scalar | \(\text{Var}(aX) = a^2\,\text{Var}(X)\) | Doubling \(X\) quadruples spread |
| Shift | \(\text{Var}(X + b) = \text{Var}(X)\) | Adding a constant shifts but doesn't stretch |
| Linear combo | \(\text{Var}(aX + b) = a^2\,\text{Var}(X)\) | Combines the two rules above |
| Sum | \(\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y) + 2\,\text{Cov}(X,Y)\) | Spreads add, with covariance correction |
| Independent sum | \(\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y)\) | When \(X \perp Y\), \(\text{Cov}=0\) |
7. The Normal Distribution
The normal (Gaussian) distribution is the workhorse of econometrics. If \(X \sim N(\mu, \sigma^2)\):
Key properties: - Symmetric, bell-shaped, centred at \(\mu\). - Fully characterised by just two parameters: mean \(\mu\) and variance \(\sigma^2\). - \(E(X) = \mu\), \(\text{Var}(X) = \sigma^2\).
The 68–95–99.7 Rule
Standardisation
Any \(N(\mu, \sigma^2)\) random variable can be transformed to the standard normal \(Z \sim N(0,1)\):
This is why z-scores from Lesson M01.L01 appear here: standardising a sample observation is the sample-level analogue of this population-level transformation. Standard normal probabilities are tabulated in every econometrics textbook.
8. The \(t\)-Distribution
When \(X \sim N(\mu, \sigma^2)\) but \(\sigma^2\) is unknown and must be estimated from data, we can no longer use the \(Z\) statistic. Instead:
The \(t\)-distribution with \(\nu\) degrees of freedom is: - Symmetric and bell-shaped like the normal, but with heavier tails. - As \(\nu \to \infty\), \(t_\nu \to N(0,1)\). - Used for all OLS coefficient hypothesis tests: \(t = \hat{\beta}/\text{se}(\hat{\beta})\).
In practice with econometric samples (\(n > 120\)), the \(t\) and \(Z\) critical values are nearly identical.
9. The Chi-Squared Distribution (\(\chi^2\))
If \(Z_1, \ldots, Z_k \overset{iid}{\sim} N(0,1)\), then:
The \(\chi^2_k\) distribution: - Is right-skewed (always \(\geq 0\)). - Has mean \(k\) and variance \(2k\). - Arises naturally when estimating variance: \((n-1)s^2/\sigma^2 \sim \chi^2_{n-1}\). - Used for Breusch–Pagan heteroskedasticity tests and goodness-of-fit tests.
10. The F-Distribution
If \(W_1 \sim \chi^2_{d_1}\) and \(W_2 \sim \chi^2_{d_2}\) are independent:
The F-distribution with numerator degrees of freedom \(d_1\) and denominator \(d_2\): - Is right-skewed, always \(\geq 0\). - Arises when testing multiple hypotheses simultaneously in OLS. - The overall F-test in regression: "Are any regressors jointly significant?"
Key relationship: \(t_\nu^2 = F_{1,\nu}\). A squared \(t\)-statistic with \(\nu\) df equals an F-statistic with \((1, \nu)\) df.
Worked Example
Standardising a Normal Variable & Finding Probabilities
The hourly wage of a randomly selected Australian full-time worker is normally distributed with mean \(\mu = \$38\) and standard deviation \(\sigma = \$12\).
Question: What proportion of workers earn between $26 and $56 per hour?
Step 1 — Standardise the bounds:
Step 2 — Find probabilities using the standard normal CDF \(\Phi\):
From tables (or R: pnorm(1.5) - pnorm(-1)):
Interpretation: About 77.5% of Australian full-time workers earn between $26 and $56 per hour under this model.
Step 3 — Variance property check:
If we double all wages (\(V = 2W\)): $\(E(V) = 2 \times 38 = \$76, \qquad \text{Var}(V) = 4 \times 144 = 576 \quad \Rightarrow \quad \sigma_V = \$24\)$
The standard deviation also doubled — confirming \(\text{Var}(aX) = a^2\text{Var}(X)\).
Common Misconception
"Economic variables are normally distributed — or if not, I should use a different method."
The normal distribution is central to econometric theory because of the Central Limit Theorem (CLT): as sample size \(n\) grows, the sampling distribution of the sample mean \(\bar{X}\) and OLS estimators \(\hat{\beta}\) converge to normal, regardless of the distribution of the underlying data.
This means: 1. You do not need wages, prices, or GDP to be normally distributed. 2. Inference (t-tests, F-tests) is justified in large samples even when residuals are skewed or heavy-tailed. 3. In small samples, however, the CLT has not kicked in, and non-normal residuals can distort inference. This is why residual diagnostics matter.
Many economic variables are inherently skewed (income, house prices, firm sizes follow Pareto or log-normal distributions). The log transformation \(\ln(Y)\) often produces a more symmetric variable and is used extensively in econometrics — not to "fix" normality, but because log-linear models have natural economic interpretations (elasticities).
Practice Prompts
Prompt 1
Let \(X\) be the annual return (%) on an Australian equity fund, with \(X \sim N(8, 25)\) (mean 8%, variance 25, so \(\sigma = 5\%\)).
(a) What is the probability of a negative return in any given year? (b) What return corresponds to the 95th percentile?
Answer
**(a)** $P(X < 0)$: standardise: $$Z = \frac{0 - 8}{5} = -1.6$$ $$P(X < 0) = \Phi(-1.6) = 1 - \Phi(1.6) = 1 - 0.9452 = \mathbf{0.0548}$$ About 5.5% chance of a negative return each year. **(b)** The 95th percentile of $N(0,1)$ is $z_{0.95} = 1.645$. $$X_{0.95} = \mu + z_{0.95} \cdot \sigma = 8 + 1.645 \times 5 = 8 + 8.225 = \mathbf{16.225\%}$$ A return of approximately 16.2% is at the 95th percentile. *In R:* `qnorm(0.95, mean=8, sd=5)` gives the same answer.Prompt 2
A discrete random variable \(X\) represents the number of Reserve Bank of Australia rate cuts in a given year, with the following distribution:
| \(x\) | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| \(P(X=x)\) | 0.30 | 0.40 | 0.20 | 0.10 |
(a) Compute \(E(X)\). (b) Compute \(\text{Var}(X)\) using the formula \(\text{Var}(X) = E(X^2) - [E(X)]^2\). (c) If each rate cut reduces the mortgage rate by 0.25%, and \(Y = 0.25X\) represents the total rate reduction, find \(E(Y)\) and \(\text{Var}(Y)\).
Answer
**(a) Expected value:** $$E(X) = 0(0.30) + 1(0.40) + 2(0.20) + 3(0.10) = 0 + 0.40 + 0.40 + 0.30 = \mathbf{1.10}$$ **(b) Variance:** First compute $E(X^2) = 0^2(0.30) + 1^2(0.40) + 2^2(0.20) + 3^2(0.10) = 0 + 0.40 + 0.80 + 0.90 = 2.10$. $$\text{Var}(X) = E(X^2) - [E(X)]^2 = 2.10 - (1.10)^2 = 2.10 - 1.21 = \mathbf{0.89}$$ **(c) Linear transformation $Y = 0.25X$:** $$E(Y) = 0.25 \times E(X) = 0.25 \times 1.10 = \mathbf{0.275\%}$$ $$\text{Var}(Y) = (0.25)^2 \times \text{Var}(X) = 0.0625 \times 0.89 = \mathbf{0.0556}$$ The expected total mortgage rate reduction is 0.275%, with standard deviation $\sqrt{0.0556} \approx 0.236\%$.Prompt 3
Explain in your own words: (a) Why does the \(t\)-distribution have heavier tails than the standard normal? (b) In a regression with \(n = 50\) observations and \(k = 4\) regressors (including the intercept), what are the degrees of freedom for the \(t\)-test on each coefficient? For the overall \(F\)-test?
Answer
**(a)** The $t$-statistic is $(\bar{X} - \mu) / (s/\sqrt{n})$. Unlike the $Z$-statistic, the denominator $s$ is itself a random variable — it varies from sample to sample. When $s$ happens to be small (underestimating $\sigma$), the $t$-statistic is inflated. This additional uncertainty from estimating $\sigma$ makes extreme values of $t$ more probable than extreme values of $Z$, producing heavier tails. As $n \to \infty$, $s \to \sigma$ and the extra uncertainty vanishes — so $t_\nu \to N(0,1)$. **(b)** With $n = 50$ observations and $k = 4$ parameters (including intercept): - **$t$-test degrees of freedom:** $n - k = 50 - 4 = \mathbf{46}$ df. Each individual coefficient is tested against $t_{46}$. - **$F$-test degrees of freedom:** The overall $F$-test has numerator df $= k - 1 = 3$ (the 3 slope coefficients, excluding the intercept) and denominator df $= n - k = 46$. So $F \sim F_{3, 46}$. *Practical note:* With 46 df, the $t_{46}$ critical value at the 5% level is approximately 2.013, very close to the normal critical value of 1.960.Further Resources
- Wooldridge, Introductory Econometrics (7th ed.): Appendix B (Fundamentals of Probability), Appendix C (Fundamentals of Mathematical Statistics) — especially B.4 on the normal distribution and C.2 on the CLT
- ECMT1020 Notes (USYD): Weeks 1–3 (R.2–R.4 on discrete RVs; R.8 on continuous distributions including normal, \(t\), \(F\), \(\chi^2\))
- Khan Academy — Probability Distributions: khanacademy.org/math/statistics-probability/random-variables-stats-library
- R Reference:
- Normal:
dnorm(),pnorm(),qnorm(),rnorm() - \(t\):
dt(),pt(),qt() - \(\chi^2\):
dchisq(),pchisq(),qchisq() - \(F\):
df(),pf(),qf() - Seeing Theory (Brown University): seeing-theory.brown.edu — interactive visualisations of probability distributions and the CLT