Probability

OpenIntro Statistics · Ch. 3 · openintro.org/book/os

Probability quantifies uncertainty. The rules are simple: probabilities are between 0 and 1, they add up for disjoint events, and conditional probability lets you update beliefs with new evidence. Bayes' theorem ties it all together.

Probability rules

For any event A: 0 ≤ P(A) ≤ 1. For disjoint events: P(A or B) = P(A) + P(B). For non-disjoint events: P(A or B) = P(A) + P(B) - P(A and B). The complement rule: P(not A) = 1 - P(A).

Scheme

; Probability rules
; P(A or B) = P(A) + P(B) - P(A and B)

(define p-a 0.4)    ; P(rain)
(define p-b 0.3)    ; P(traffic)
(define p-ab 0.12)  ; P(rain AND traffic)

; Addition rule (general)
(define p-a-or-b (+ p-a p-b (- p-ab)))
(display "P(rain OR traffic) = ")
(display p-a-or-b) (newline)

; Complement rule
(define p-not-a (- 1 p-a))
(display "P(no rain) = ")
(display p-not-a) (newline)

; If disjoint (mutually exclusive), P(A and B) = 0
(define p-heads 0.5)
(define p-tails 0.5)
(display "P(heads OR tails) = ")
(display (+ p-heads p-tails))

Conditional probability

P(A | B) = P(A and B) / P(B). The probability of A, given that B happened. This is how we update probabilities with evidence.

Scheme

; Conditional probability: P(A|B) = P(A and B) / P(B)

(define p-disease 0.01)         ; prevalence
(define p-positive-given-disease 0.95)  ; sensitivity
(define p-positive-given-healthy 0.05)  ; false positive rate

; P(positive) by total probability
(define p-healthy (- 1 p-disease))
(define p-positive
  (+ (* p-positive-given-disease p-disease)
     (* p-positive-given-healthy p-healthy)))

(display "P(positive test) = ")
(display p-positive) (newline)

; P(disease | positive) by Bayes' theorem
(define p-disease-given-positive
  (/ (* p-positive-given-disease p-disease) p-positive))

(display "P(disease | positive) = ")
(display (exact->inexact p-disease-given-positive)) (newline)

; Even with a 95% accurate test, a positive result
; only means ~16% chance of disease when prevalence is 1%
(display "Base rate matters.")

Independence

Events A and B are independent if P(A | B) = P(A), equivalently P(A and B) = P(A) * P(B). Coin flips are independent. Test results from the same patient are not.

Scheme

; Independence check: P(A and B) =? P(A) * P(B)

; Two coin flips: independent
(define p-h1 0.5)  ; first heads
(define p-h2 0.5)  ; second heads
(define p-both-heads (* p-h1 p-h2))
(display "P(both heads) = ")
(display p-both-heads) (newline)
(display "P(H1) * P(H2) = ")
(display (* p-h1 p-h2)) (newline)
(display "Independent? ")
(display (= p-both-heads (* p-h1 p-h2))) (newline)
(newline)

; Rain and umbrella: NOT independent
(define p-rain 0.3)
(define p-umbrella 0.4)
(define p-rain-and-umbrella 0.25)  ; higher than product
(display "P(rain) * P(umbrella) = ")
(display (* p-rain p-umbrella)) (newline)
(display "P(rain AND umbrella)  = ")
(display p-rain-and-umbrella) (newline)
(display "Independent? ")
(display (= p-rain-and-umbrella (* p-rain p-umbrella)))

Neighbors

Related chapters

🎰 Grinstead Ch.4 — conditional probability with combinatorial foundations
📡 Shannon Ch.3 — probability as the basis for entropy and information

Foundations (Wikipedia)

Translation notes

OpenIntro introduces probability with contingency tables and tree diagrams. We focus on the computational rules. The Bayes' theorem example (disease screening) is a standard illustration of base rate neglect. The original also covers probability trees and more complex multi-step scenarios. For rigorous probability axioms, see 🎰 Grinstead Ch. 1.

Want the full treatment? Read OpenIntro Statistics, Ch. 3.

← Summarizing Data by june.kim Distributions →