Time Series Analysis

MIT OCW 18.S096 + 15.450 (CC BY-NC-SA 4.0)

Time series models decompose a sequence of observations into signal and noise. AR models use past values to predict the future; stationarity determines whether those predictions are meaningful. Cointegration finds stable long-run relationships between non-stationary series.

Autoregressive (AR) models

An AR(1) process: y(t) = c + phi * y(t-1) + epsilon(t). If |phi| < 1, the process mean-reverts to c/(1-phi). Phi near 1 means high persistence; phi near 0 means each observation is mostly noise. Negative phi produces alternating overshoots.

Scheme

; AR(1): y(t) = c + phi * y(t-1) + epsilon(t)
; Simulate with fixed seed for reproducibility

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (ar1-simulate c phi sigma n seed)
  (let ((mu (/ c (- 1 phi))))
    (define (go i y acc seed-val)
      (if (= i n) (reverse acc)
          (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
                 (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
                 (eps (* sigma (box-muller u1 u2)))
                 (y-new (+ c (* phi y) eps)))
            (go (+ i 1) y-new (cons y-new acc)
                (modulo (* seed-val 16807) 2147483647)))))
    (go 0 mu (list mu) seed)))

; phi=0.8, c=2, so long-run mean = 2/(1-0.8) = 10
(define path (ar1-simulate 2.0 0.8 1.0 10 42))
(display "AR(1) path (c=2, phi=0.8, mean=10):") (newline)
(for-each (lambda (y) (display (/ (round (* y 100)) 100)) (display " ")) path)
(newline) (newline)
(display "Long-run mean: ") (display (/ 2.0 (- 1 0.8)))

; AR(1): y(t) = c + phi * y(t-1) + epsilon(t)
; Simulate with fixed seed for reproducibility

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (ar1-simulate c phi sigma n seed)
  (let ((mu (/ c (- 1 phi))))
    (define (go i y acc seed-val)
      (if (= i n) (reverse acc)
          (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
                 (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
                 (eps (* sigma (box-muller u1 u2)))
                 (y-new (+ c (* phi y) eps)))
            (go (+ i 1) y-new (cons y-new acc)
                (modulo (* seed-val 16807) 2147483647)))))
    (go 0 mu (list mu) seed)))

; phi=0.8, c=2, so long-run mean = 2/(1-0.8) = 10
(define path (ar1-simulate 2.0 0.8 1.0 10 42))
(display "AR(1) path (c=2, phi=0.8, mean=10):") (newline)
(for-each (lambda (y) (display (/ (round (* y 100)) 100)) (display " ")) path)
(newline) (newline)
(display "Long-run mean: ") (display (/ 2.0 (- 1 0.8)))

Stationarity and unit roots

A time series is stationary if its mean and variance do not change over time. When phi = 1, the process is a random walk — it has a unit root and is non-stationary. Stock prices are typically non-stationary; log returns are stationary. The Dickey-Fuller test checks whether phi is significantly less than 1.

Scheme

; Compare stationary (phi=0.8) vs random walk (phi=1.0)
; Variance of AR(1) = sigma^2 / (1 - phi^2)
; Random walk variance grows without bound

(define (ar1-variance sigma phi)
  (if (>= (abs phi) 1.0)
      (display "infinite (non-stationary)")
      (/ (* sigma sigma) (- 1 (* phi phi)))))

(display "phi=0.8, sigma=1:  var = ")
(display (/ (round (* (ar1-variance 1.0 0.8) 10000)) 10000))
(newline)

(display "phi=0.5, sigma=1:  var = ")
(display (/ (round (* (ar1-variance 1.0 0.5) 10000)) 10000))
(newline)

(display "phi=0.95, sigma=1: var = ")
(display (/ (round (* (ar1-variance 1.0 0.95) 10000)) 10000))
(newline)

(display "phi=1.0, sigma=1:  var = ")
(ar1-variance 1.0 1.0)
(newline)

(newline)
(display "As phi -> 1, variance explodes. At phi=1, no stationary distribution exists.")

Moving averages and ARMA

An MA(1) process: y(t) = mu + epsilon(t) + theta * epsilon(t-1). Past shocks persist for exactly one period. Combine AR and MA to get ARMA(p,q): p autoregressive lags and q moving-average lags. ARMA is parsimonious — a low-order ARMA often fits as well as a high-order pure AR.

Scheme

; MA(1): y(t) = mu + eps(t) + theta * eps(t-1)
; ARMA(1,1): y(t) = c + phi*y(t-1) + eps(t) + theta*eps(t-1)

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (arma11-simulate c phi theta sigma n seed)
  (define (go i y prev-eps acc seed-val)
    (if (= i n) (reverse acc)
        (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
               (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
               (eps (* sigma (box-muller u1 u2)))
               (y-new (+ c (* phi y) eps (* theta prev-eps))))
          (go (+ i 1) y-new eps (cons y-new acc)
              (modulo (* seed-val 16807) 2147483647)))))
  (go 0 0 0 '() seed))

(define path (arma11-simulate 1.0 0.7 0.3 0.5 10 42))
(display "ARMA(1,1) path (c=1, phi=0.7, theta=0.3):") (newline)
(for-each (lambda (y) (display (/ (round (* y 100)) 100)) (display " ")) path)
(newline) (newline)

; Autocorrelation of MA(1): rho(1) = theta / (1 + theta^2)
(define theta 0.3)
(define rho1 (/ theta (+ 1 (* theta theta))))
(display "MA(1) autocorrelation at lag 1: ")
(display (/ (round (* rho1 10000)) 10000))
(newline)
(display "MA(1) autocorrelation at lag 2+: 0 (sharp cutoff)")

; MA(1): y(t) = mu + eps(t) + theta * eps(t-1)
; ARMA(1,1): y(t) = c + phi*y(t-1) + eps(t) + theta*eps(t-1)

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (arma11-simulate c phi theta sigma n seed)
  (define (go i y prev-eps acc seed-val)
    (if (= i n) (reverse acc)
        (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
               (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
               (eps (* sigma (box-muller u1 u2)))
               (y-new (+ c (* phi y) eps (* theta prev-eps))))
          (go (+ i 1) y-new eps (cons y-new acc)
              (modulo (* seed-val 16807) 2147483647)))))
  (go 0 0 0 '() seed))

(define path (arma11-simulate 1.0 0.7 0.3 0.5 10 42))
(display "ARMA(1,1) path (c=1, phi=0.7, theta=0.3):") (newline)
(for-each (lambda (y) (display (/ (round (* y 100)) 100)) (display " ")) path)
(newline) (newline)

; Autocorrelation of MA(1): rho(1) = theta / (1 + theta^2)
(define theta 0.3)
(define rho1 (/ theta (+ 1 (* theta theta))))
(display "MA(1) autocorrelation at lag 1: ")
(display (/ (round (* rho1 10000)) 10000))
(newline)
(display "MA(1) autocorrelation at lag 2+: 0 (sharp cutoff)")

Cointegration

Two non-stationary series are cointegrated if a linear combination of them is stationary. Stock prices of Coca-Cola and Pepsi each wander randomly, but their spread mean-reverts. Pairs trading exploits this: when the spread widens, bet on convergence. The Engle-Granger test runs a regression and checks the residuals for stationarity.

Scheme

; Cointegration: two random walks with a stationary spread
; x(t) = x(t-1) + eps_x(t)           (non-stationary)
; y(t) = beta*x(t) + z(t)             where z(t) is stationary
; spread = y(t) - beta*x(t) = z(t)    (stationary!)

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (cointegrated-pair n beta sigma-x sigma-z kappa seed)
  (define (go i x z acc-x acc-y acc-spread seed-val)
    (if (= i n) (values (reverse acc-x) (reverse acc-y) (reverse acc-spread))
        (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
               (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
               (u3 (/ (modulo (* (+ seed-val 99991) 16807) 2147483647) 2147483647.0))
               (u4 (/ (modulo (* (+ seed-val 33333) 16807) 2147483647) 2147483647.0))
               (eps-x (* sigma-x (box-muller u1 u2)))
               (eps-z (* sigma-z (box-muller u3 u4)))
               (x-new (+ x eps-x))
               (z-new (+ (* (- 1 kappa) z) eps-z))
               (y-new (+ (* beta x-new) z-new))
               (spread z-new))
          (go (+ i 1) x-new z-new
              (cons x-new acc-x) (cons y-new acc-y) (cons spread acc-spread)
              (modulo (* seed-val 16807) 2147483647)))))
  (go 0 100 0 '() '() '() seed))

(define-values (xs ys spreads) (cointegrated-pair 8 1.5 1.0 0.5 0.3 42))
(display "X (random walk):  ")
(for-each (lambda (x) (display (/ (round (* x 10)) 10)) (display " ")) xs)
(newline)
(display "Y (cointegrated): ")
(for-each (lambda (y) (display (/ (round (* y 10)) 10)) (display " ")) ys)
(newline)
(display "Spread (stationary): ")
(for-each (lambda (s) (display (/ (round (* s 100)) 100)) (display " ")) spreads)
(newline) (newline)
(display "X and Y both wander. Their spread mean-reverts.")

; Cointegration: two random walks with a stationary spread
; x(t) = x(t-1) + eps_x(t)           (non-stationary)
; y(t) = beta*x(t) + z(t)             where z(t) is stationary
; spread = y(t) - beta*x(t) = z(t)    (stationary!)

(define (box-muller u1 u2)
  (* (sqrt (* -2 (log u1)))
     (cos (* 2 3.141592653589793 u2))))

(define (cointegrated-pair n beta sigma-x sigma-z kappa seed)
  (define (go i x z acc-x acc-y acc-spread seed-val)
    (if (= i n) (values (reverse acc-x) (reverse acc-y) (reverse acc-spread))
        (let* ((u1 (/ (modulo (* seed-val 16807) 2147483647) 2147483647.0))
               (u2 (/ (modulo (* (+ seed-val 48271) 16807) 2147483647) 2147483647.0))
               (u3 (/ (modulo (* (+ seed-val 99991) 16807) 2147483647) 2147483647.0))
               (u4 (/ (modulo (* (+ seed-val 33333) 16807) 2147483647) 2147483647.0))
               (eps-x (* sigma-x (box-muller u1 u2)))
               (eps-z (* sigma-z (box-muller u3 u4)))
               (x-new (+ x eps-x))
               (z-new (+ (* (- 1 kappa) z) eps-z))
               (y-new (+ (* beta x-new) z-new))
               (spread z-new))
          (go (+ i 1) x-new z-new
              (cons x-new acc-x) (cons y-new acc-y) (cons spread acc-spread)
              (modulo (* seed-val 16807) 2147483647)))))
  (go 0 100 0 '() '() '() seed))

(define-values (xs ys spreads) (cointegrated-pair 8 1.5 1.0 0.5 0.3 42))
(display "X (random walk):  ")
(for-each (lambda (x) (display (/ (round (* x 10)) 10)) (display " ")) xs)
(newline)
(display "Y (cointegrated): ")
(for-each (lambda (y) (display (/ (round (* y 10)) 10)) (display " ")) ys)
(newline)
(display "Spread (stationary): ")
(for-each (lambda (s) (display (/ (round (* s 100)) 100)) (display " ")) spreads)
(newline) (newline)
(display "X and Y both wander. Their spread mean-reverts.")

Neighbors

📉 Finance 2 Ch.5 — GARCH is a volatility-specific time series model
📊 Statistics Ch.4 — regression foundations for AR estimation
🎲 Probability Ch.5 — Markov chains share the AR memory structure

← Volatility Modeling · 5 by june.kim Risk Management · 7 →