Decision Making

Lovelace textbook · CC BY-SA 4.0 · computationalcognitivescience.github.io/lovelace/home

Rational agents maximize expected utility. Real humans don't. Kahneman and Tversky's prospect theory explains the gap: people weight losses more than gains, overweight small probabilities, and evaluate outcomes relative to a reference point. Bounded rationality reframes the question: given limited time and information, satisficing often beats optimizing.

Expected utility

The normative theory: choose the option with the highest expected utility. EU = sum of probability times utility for each outcome. This works when you have perfect information, unlimited computation, and stable preferences. Violations of EU theory are not mistakes to be corrected. They are data about how human cognition actually works.

Scheme

; Expected utility: EU = sum of p(outcome) * u(outcome)
;
; Gamble A: 80% chance of $4000, 20% chance of $0
; Gamble B: 100% chance of $3000
; EU(A) = 0.8 * 4000 = 3200
; EU(B) = 1.0 * 3000 = 3000
; A rational agent picks A. Most people pick B.

(define (expected-utility outcomes)
  (apply + (map (lambda (pair) (* (car pair) (cdr pair))) outcomes)))

(define gamble-a (list (cons 0.8 4000) (cons 0.2 0)))
(define gamble-b (list (cons 1.0 3000)))

(display "EU(Gamble A) = ") (display (expected-utility gamble-a)) (newline)
(display "EU(Gamble B) = ") (display (expected-utility gamble-b)) (newline)
(display "EU theory picks A, most people pick B (certainty effect)")

Prospect theory

Prospect theory replaces utility with a value function that is concave for gains, convex for losses, and steeper for losses than gains (loss aversion). It also replaces probabilities with decision weights that overweight small probabilities and underweight large ones. The reference point makes the same outcome feel like a gain or a loss depending on framing.

Scheme

; Prospect theory value function
; v(x) = x^0.88 for gains
; v(x) = -2.25 * |x|^0.88 for losses
; Loss aversion coefficient = 2.25

(define (prospect-value x)
  (if (>= x 0)
      (expt x 0.88)
      (* -2.25 (expt (abs x) 0.88))))

(display "v(100)  = ") (display (prospect-value 100)) (newline)
(display "v(-100) = ") (display (prospect-value -100)) (newline)
(display "Loss hurts ~2.25x more than equivalent gain feels good") (newline)
(newline)

; The framing effect:
; Option 1: sure gain of $50
; Option 2: sure loss of $50
; Same magnitude, very different subjective values
(display "v(50)  = ") (display (prospect-value 50)) (newline)
(display "v(-50) = ") (display (prospect-value -50)) (newline)
(display "|v(-50)| / v(50) = ")
(display (/ (abs (prospect-value -50)) (prospect-value 50)))

Bounded rationality and satisficing

Herbert Simon argued that real decision-makers face limited time, limited information, and limited computation. Bounded rationality asks: given these constraints, what decision procedures perform well? Satisficing means searching until you find an option that meets a threshold, then stopping. This is structurally similar to competitive inhibition: a winner-take-all gate that passes the first signal above threshold and suppresses the rest. It often outperforms optimization in uncertain environments because it avoids overfitting to noisy estimates.

Scheme

; Satisficing vs optimizing
; Scenario: hiring. You interview candidates sequentially.
; Optimizer: interview all N, pick the best.
; Satisficer: set a threshold, hire the first who meets it.
;
; The "secretary problem": optimal stopping rule is to
; observe the first N/e candidates, then hire the next one
; better than all observed so far.

(define e 2.718)

(define (optimal-stopping-threshold n)
  (inexact->exact (round (/ n e))))

(display "With 100 candidates:") (newline)
(display "  Observe first ")
(display (optimal-stopping-threshold 100))
(display " candidates (the 'look' phase)") (newline)
(display "  Then hire next one better than all seen") (newline)
(newline)
(display "This satisficing rule finds the best candidate")
(newline)
(display "  ~37% of the time — surprisingly good for") (newline)
(display "  a strategy that ignores most of the data.")

Notation reference

Term	Meaning
EU	Expected utility: sum of p * u(outcome)
v(x)	Prospect theory value function
Loss aversion	Losses loom ~2.25x larger than equivalent gains
Satisficing	Accept first option above threshold
N/e rule	Optimal stopping: observe 37%, then decide

Neighbors

Lovelace Ch.5 — reinforcement learning as sequential decision making
Game theory pages — strategic decision making with multiple agents
Prospect theory
Bounded rationality

Translation notes

The Lovelace textbook covers prospect theory's probability weighting function in addition to the value function, and discusses framing effects, the endowment effect, and the status quo bias as downstream consequences. This page focuses on the three main ideas: EU as normative baseline, prospect theory as descriptive alternative, and bounded rationality as a different framing of the question. The textbook also covers multi-attribute decision making and decision under uncertainty.

Read the original: Lovelace, Chapter 6.

← Reinforcement Learning by june.kim Language and Communication · 7 of 8 →