Conditional Probability

Grinstead & Snell · GFDL · PDF

Conditional probability is the probability of A given that B has occurred: P(A|B) = P(A ∩ B) / P(B). This is how you update beliefs with new information. Bayes' theorem flips the direction: knowing P(B|A), compute P(A|B).

P(A|B) — restricting the sample space

When you learn that B has occurred, the sample space shrinks from Ω to B. The conditional probability P(A|B) re-normalizes by dividing by P(B). Only outcomes in both A and B survive.

Scheme

; Conditional probability: P(A|B) = P(A and B) / P(B)

(define (cond-prob p-a-and-b p-b)
  (/ p-a-and-b p-b))

; Example: fair die. A = "even", B = "greater than 3"
; Omega = (1 2 3 4 5 6)
; A = (2 4 6), B = (4 5 6), A and B = (4 6)
(define p-a 3/6)
(define p-b 3/6)
(define p-a-and-b 2/6)

(display "P(A) = ") (display (exact->inexact p-a)) (newline)
(display "P(B) = ") (display (exact->inexact p-b)) (newline)
(display "P(A and B) = ") (display (exact->inexact p-a-and-b)) (newline)
(display "P(A|B) = P(even | >3) = ")
(display (exact->inexact (cond-prob p-a-and-b p-b)))
; 2/3 — knowing the roll is >3 makes "even" more likely

The multiplication rule

Rearranging the definition: P(A ∩ B) = P(A|B) · P(B). For a chain of events: P(A ∩ B ∩ C) = P(A) · P(B|A) · P(C|A ∩ B). Multiply along the branches of the tree.

Scheme

; Multiplication rule: P(A and B) = P(A|B) * P(B)
; Drawing cards without replacement

; P(first card is ace) = 4/52
; P(second card is ace | first was ace) = 3/51
; P(both aces) = (4/52) * (3/51)

(define p-first-ace 4/52)
(define p-second-ace-given-first 3/51)
(define p-both-aces (* p-first-ace p-second-ace-given-first))

(display "P(1st ace) = ") (display (exact->inexact p-first-ace)) (newline)
(display "P(2nd ace | 1st ace) = ") (display (exact->inexact p-second-ace-given-first)) (newline)
(display "P(both aces) = ") (display (exact->inexact p-both-aces))

Bayes' theorem

P(A|B) = P(B|A) · P(A) / P(B). You know how likely the evidence is given the hypothesis (P(B|A)). Bayes lets you compute how likely the hypothesis is given the evidence (P(A|B)). The denominator P(B) is often expanded using the law of total probability.

Scheme

; Bayes' theorem: P(A|B) = P(B|A) * P(A) / P(B)
; Classic example: medical test

; Disease prevalence: 1%
(define p-disease 0.01)
(define p-healthy 0.99)

; Test sensitivity: P(positive | disease) = 95%
(define p-pos-given-disease 0.95)
; False positive rate: P(positive | healthy) = 5%
(define p-pos-given-healthy 0.05)

; P(positive) by total probability
(define p-pos (+ (* p-pos-given-disease p-disease)
                 (* p-pos-given-healthy p-healthy)))

; P(disease | positive) by Bayes
(define p-disease-given-pos
  (/ (* p-pos-given-disease p-disease) p-pos))

(display "P(positive) = ") (display p-pos) (newline)
(display "P(disease | positive) = ")
(display p-disease-given-pos) (newline)
; Only ~16%! Most positives are false positives
; because the disease is rare.
(display "Most positives are healthy people.")

Independence

Events A and B are independent if P(A|B) = P(A), meaning B gives no information about A. Equivalently, P(A ∩ B) = P(A) · P(B). Coin flips are independent. Card draws without replacement are not.

Scheme

; Independence: P(A and B) = P(A) * P(B)

(define (independent? p-a p-b p-a-and-b)
  (let ((product (* p-a p-b)))
    (< (abs (- p-a-and-b product)) 0.0001)))

; Two fair coins: P(H1) = 1/2, P(H2) = 1/2, P(H1 and H2) = 1/4
(display "coins independent? ")
(display (independent? 0.5 0.5 0.25)) (newline)

; Die: A = "even", B = "> 3"
; P(A) = 1/2, P(B) = 1/2, P(A and B) = 2/6 = 1/3
(display "even and >3 independent? ")
(display (independent? 0.5 0.5 (/ 2.0 6))) (newline)
; Not independent: knowing >3 changes P(even) from 1/2 to 2/3

; Die: A = "even", C = "< 5"
; P(A) = 1/2, P(C) = 4/6 = 2/3, P(A and C) = 2/6 = 1/3
(display "even and <5 independent? ")
(display (independent? 0.5 (/ 4.0 6) (/ 2.0 6)))
; Independent: P(A)*P(C) = 1/2 * 2/3 = 1/3 = P(A and C)

Notation reference

Textbook	Scheme	Meaning
P(A\|B)	(cond-prob p-ab p-b)	Conditional probability
P(A ∩ B) = P(A\|B)P(B)	(* p-a-given-b p-b)	Multiplication rule
P(A\|B) = P(B\|A)P(A)/P(B)	(/ (* p-ba p-a) p-b)	Bayes' theorem
A ⊥ B	(independent? ...)	Independence
P(B) = ∑ P(B\|A_i)P(A_i)	(+ (* p-b-a1 p-a1) ...)	Law of total probability

Neighbors

Adjacent chapters

🎰 Ch 3: Combinatorics — counting makes conditional probabilities computable
🎰 Ch 1: Discrete Probability — the axioms that conditional probability builds on
🎰 Ch 2: Continuous Probability — conditioning works for densities too (condition on events with positive measure)
🧠 Cognitive Science Ch.2 — Bayes' theorem as a model of human reasoning
🤖 ML Ch.4 — logistic regression uses conditional probability directly
📊 Statistics Ch.3 — conditional probability in the context of inference

Paper pages

🍞 Fritz 2020 — Markov categories axiomatize conditional probability as morphism composition. Conditioning is a morphism, not a derived operation
🍞 Baez-Fritz 2011 — entropy measures how much conditioning reduces uncertainty
🍞 Staton 2025 — probabilistic programs implement conditioning via observe statements

Related foundations

🧠 Lovelace Ch.2 Bayesian Inference — Bayes' theorem applied to cognitive science

Foundations (Wikipedia)

Ready for the real thing? Read Ch 4 of Grinstead & Snell.

← Combinatorics by june.kim Distributions →