Optimal Control

Wikipedia · Linear-quadratic regulator · CC BY-SA 4.0

Optimal control finds the input that minimizes a cost function. The LQR (linear quadratic regulator) minimizes a weighted sum of state deviation and control effort. The solution comes from the Riccati equation. The tradeoff: penalize state error more for faster response, penalize effort more for smaller actuators.

The cost function

Define a cost J that sums two things over time: how far the state is from zero (weighted by matrix Q) and how much control effort you use (weighted by matrix R). J = integral of (x'Qx + u'Ru) dt. Larger Q means "care more about tracking." Larger R means "care more about saving energy." The optimal control law is a linear gain: u = -Kx, where K is computed from the Riccati equation.

LQR: the linear case

For a linear system x' = Ax + Bu with quadratic cost, the optimal gain K = R^(-1) B' P where P solves the algebraic Riccati equation: A'P + PA - PBR^(-1)B'P + Q = 0. The result is a constant-gain state feedback law. No online optimization needed: compute K once, apply u = -Kx forever.

Scheme

; LQR-like simulation: u = -K*x for a double integrator
; x1' = x2, x2' = u
; Hand-tuned K = [k1, k2] (in practice, solve Riccati)

(define k1 1.0) (define k2 1.5)  ; feedback gains
(define dt 0.05)

(define (lqr-sim x1 x2 steps)
  (if (= steps 0) (begin (display "Done.") (newline))
    (let* ((u (- (+ (* k1 x1) (* k2 x2))))  ; u = -Kx
           (new-x1 (+ x1 (* x2 dt)))
           (new-x2 (+ x2 (* u dt)))
           (cost (+ (* x1 x1) (* 0.1 u u)))) ; instantaneous cost
      (if (= 0 (modulo steps 10))
        (begin
          (display "  x1=") (display new-x1)
          (display "  x2=") (display new-x2)
          (display "  u=") (display u)
          (display "  cost=") (display cost)
          (newline))
        'skip)
      (lqr-sim new-x1 new-x2 (- steps 1)))))

(display "LQR-like control of double integrator:") (newline)
(lqr-sim 5.0 0.0 100)

Neighbors

∫ Calculus Ch. 12 — Optimization — LQR is optimization over function space

← State Space by june.kim Kalman Filter →