Skip to content

Lesson M01.L02: Probability Fundamentals

Field Value
Module M01 — Probability & Statistics Review
Level Graduate (ECMT5001)
Duration ~60 minutes
Learning Objective Understand probability axioms, conditional probability, independence, and Bayes' theorem, and apply them to economic and policy questions.
Provenance Wooldridge Appendix B.1; ECMT1020 Week 1 (distributions and probability)

Explanation

1. Sample Spaces and Events

An experiment is any process with uncertain outcomes. The sample space \(\Omega\) is the set of all possible outcomes.

Examples: - Tossing a coin: \(\Omega = \{H, T\}\) - Rolling a die: \(\Omega = \{1,2,3,4,5,6\}\) - Australian economy next year: \(\Omega = \{\text{recession}, \text{stagnation}, \text{growth}\}\)

An event \(A\) is any subset of \(\Omega\). For example, "the economy grows" is the event \(\{growth\}\).

2. Probability Axioms

A probability measure assigns a number \(P(A)\) to each event \(A\). Three axioms (Kolmogorov, 1933) define valid probability:

Axiom 1 — Non-negativity: \(P(A) \geq 0\) for all events \(A\).

Axiom 2 — Normalisation: \(P(\Omega) = 1\) (something must happen).

Axiom 3 — Countable additivity: For mutually exclusive events \(A\) and \(B\) (i.e., \(A \cap B = \emptyset\)):

\[P(A \cup B) = P(A) + P(B)\]

Everything else in probability follows from these three axioms. Key consequences:

\[P(\emptyset) = 0, \qquad P(A^c) = 1 - P(A), \qquad P(A \cup B) = P(A) + P(B) - P(A \cap B)\]

3. Conditional Probability

Conditional probability is the probability of event \(A\) occurring given that event \(B\) has already occurred:

\[P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \qquad P(B) > 0\]

This can be rearranged to give the multiplication rule:

\[P(A \cap B) = P(A \mid B) \cdot P(B) = P(B \mid A) \cdot P(A)\]

Economic interpretation: "What is the probability that the RBA raises rates, given that inflation exceeded 4%?" is a conditional probability. We are restricting attention to the sub-world where inflation exceeded 4%.

4. Independence

Events \(A\) and \(B\) are independent if knowing that \(B\) occurred gives no information about whether \(A\) occurred:

\[P(A \mid B) = P(A) \qquad \Longleftrightarrow \qquad P(A \cap B) = P(A)\,P(B)\]

Important: Independence is a symmetric relationship — if \(A\) is independent of \(B\), then \(B\) is independent of \(A\).

Independence is the key assumption in random sampling. When we draw \(n\) observations independently, the joint probability of the entire sample is just the product of individual probabilities.

Caution: Independence is an assumption, not something directly observable from data. Correlation equal to zero is necessary but not sufficient for independence.

5. Total Probability Rule

If events \(B_1, B_2, \ldots, B_k\) are mutually exclusive and exhaustive (they partition \(\Omega\)), then for any event \(A\):

\[P(A) = \sum_{j=1}^{k} P(A \mid B_j)\,P(B_j)\]

This rule allows us to compute an unconditional probability by averaging over conditioning scenarios.

Example: Let \(B_1 =\) recession, \(B_2 =\) growth. Then:

\[P(\text{RBA cuts rates}) = P(\text{cut} \mid \text{recession})\,P(\text{recession}) + P(\text{cut} \mid \text{growth})\,P(\text{growth})\]

6. Bayes' Theorem

Bayes' theorem reverses a conditional probability. Starting from the multiplication rule:

\[P(A \cap B) = P(B \mid A)\,P(A) = P(A \mid B)\,P(B)\]

Rearranging:

\[\boxed{P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}}\]

Using the total probability rule to expand \(P(B)\) across a partition \(\{A, A^c\}\):

\[P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B \mid A)\,P(A) + P(B \mid A^c)\,P(A^c)}\]

Terminology: - \(P(A)\) is the prior — your belief before observing \(B\). - \(P(A \mid B)\) is the posterior — your updated belief after observing \(B\). - \(P(B \mid A)\) is the likelihood — how probable is the evidence if \(A\) is true?

Bayes' theorem formalises how rational agents update beliefs in the face of new evidence. In econometrics, this underpins Bayesian inference and calibrated forecasting.

7. The Prosecutor's Fallacy (Confusing \(P(A|B)\) with \(P(B|A)\))

A critical practical error is confusing the direction of conditioning. \(P(A \mid B) \neq P(B \mid A)\) in general.

Economic example: - \(P(\text{recession} \mid \text{negative GDP last quarter}) = 0.70\) — high probability of recession given negative GDP. - \(P(\text{negative GDP last quarter} \mid \text{recession}) = 0.85\) — most recessions do show negative GDP.

These are very different quantities. Confusing them leads to over-confident forecasts.

Worked Example

Bayes' Theorem: Leading Indicator & Recession Forecast

The Reserve Bank of Australia uses a Leading Economic Indicator (LEI) to forecast recessions. Suppose:

  • Historical data suggest Australia enters recession in 15% of years: \(P(\text{Rec}) = 0.15\)
  • When a recession does occur, the LEI signals "negative" 80% of the time: \(P(\text{LEI}^- \mid \text{Rec}) = 0.80\)
  • When there is no recession, the LEI still fires a false alarm 20% of the time: \(P(\text{LEI}^- \mid \text{No Rec}) = 0.20\)

Question: If the LEI signals negative today, what is the probability Australia is actually in a recession?

Solution using Bayes' theorem:

Let \(R =\) "recession", \(L^- =\) "LEI signals negative".

Step 1 — Prior probability of recession: $\(P(R) = 0.15, \qquad P(R^c) = 0.85\)$

Step 2 — Total probability of a negative LEI signal: $\(P(L^-) = P(L^- \mid R)\,P(R) + P(L^- \mid R^c)\,P(R^c)\)$ $\(= (0.80)(0.15) + (0.20)(0.85)\)$ $\(= 0.12 + 0.17 = 0.29\)$

Step 3 — Apply Bayes' theorem: $\(P(R \mid L^-) = \frac{P(L^- \mid R)\,P(R)}{P(L^-)} = \frac{(0.80)(0.15)}{0.29} = \frac{0.12}{0.29} \approx \mathbf{0.414}\)$

Interpretation: Even after the LEI fires, there is only a 41.4% probability of an actual recession — well below the intuitive guess of 80%. Why? Because recessions are rare (\(P(R) = 0.15\)), so most negative signals are false alarms.

This result illustrates base rate neglect: people over-weight the likelihood (\(P(L^- \mid R) = 0.80\)) and under-weight the prior (\(P(R) = 0.15\)).

Common Misconception

"\(P(A \mid B)\) and \(P(B \mid A)\) are the same thing."

They are almost never equal. Confusing the two is known as the Prosecutor's Fallacy in legal statistics, and base rate neglect in economic forecasting.

Concretely: knowing that 90% of economic crises are preceded by a banking shock does not mean that 90% of banking shocks lead to crises. There are many banking shocks (high \(P(\text{shock} \mid \text{crisis})\)), but only a small fraction escalate (low \(P(\text{crisis} \mid \text{shock})\)).

Always be explicit about which direction you are conditioning — especially when interpreting regression results, where we frequently condition on specific values of regressors.

Practice Prompts

Prompt 1

An analyst estimates: - 30% of Australian firms apply for government R&D grants: \(P(G) = 0.30\) - 70% of firms that apply are approved: \(P(\text{Approved} \mid G) = 0.70\)

(a) What is \(P(\text{Approved} \cap G)\), the probability a randomly selected firm is both an applicant and approved? (b) If approval is independent of application in a different (hypothetical) scheme, what would \(P(\text{Approved} \cap G)\) equal, given \(P(\text{Approved}) = 0.50\)?

Answer **(a)** Using the multiplication rule: $$P(\text{Approved} \cap G) = P(\text{Approved} \mid G) \cdot P(G) = 0.70 \times 0.30 = \mathbf{0.21}$$ 21% of all firms are both applicants and approved. **(b)** Under independence: $$P(\text{Approved} \cap G) = P(\text{Approved}) \cdot P(G) = 0.50 \times 0.30 = \mathbf{0.15}$$ The lower value under independence reflects that in the original scheme, applicants are *self-selected* — firms that apply are more likely to meet the criteria, so approval is *not* independent of application.

Prompt 2

A credit rating agency classifies Australian mortgage holders as: - High risk (H): 20% of borrowers - Medium risk (M): 50% - Low risk (L): 30%

Default rates: \(P(\text{default} \mid H) = 0.25\), \(P(\text{default} \mid M) = 0.08\), \(P(\text{default} \mid L) = 0.02\).

Using the total probability rule, find \(P(\text{default})\) — the overall probability that a randomly selected borrower defaults.

Answer $$P(\text{default}) = P(\text{default} \mid H)P(H) + P(\text{default} \mid M)P(M) + P(\text{default} \mid L)P(L)$$ $$= (0.25)(0.20) + (0.08)(0.50) + (0.02)(0.30)$$ $$= 0.050 + 0.040 + 0.006 = \mathbf{0.096}$$ About 9.6% of borrowers default. Note this is a weighted average of default rates, with weights equal to the share of each risk group. The result is *much* closer to the medium-risk rate than the high-risk rate, because most borrowers are not high-risk.

Prompt 3

Using the mortgage data from Prompt 2: A borrower has defaulted. Using Bayes' theorem, what is the probability they were high risk?

Answer We want $P(H \mid \text{default})$. $$P(H \mid \text{default}) = \frac{P(\text{default} \mid H) \cdot P(H)}{P(\text{default})} = \frac{(0.25)(0.20)}{0.096} = \frac{0.050}{0.096} \approx \mathbf{0.521}$$ **Interpretation:** Even though only 20% of borrowers are classified as high risk *before* a default event, once a default occurs the probability the borrower was high risk jumps to 52%. Bayes' theorem has updated our beliefs in light of the evidence. Similarly: - $P(M \mid \text{default}) = (0.08 \times 0.50)/0.096 \approx 0.417$ - $P(L \mid \text{default}) = (0.02 \times 0.30)/0.096 \approx 0.063$ These three posterior probabilities sum to 1 ✓

Further Resources