Similarity

Jim Hefferon · GFDL + CC BY-SA 2.5 · Linear Algebra

An eigenvector of a linear map is a nonzero vector whose direction is unchanged by the map: Av = lambda v. The scalar lambda is the eigenvalue. Finding eigenvalues reduces to solving det(A - lambda I) = 0. When a matrix has enough independent eigenvectors, it can be diagonalized: the map becomes just scaling along each eigenvector axis.

Eigenvalues and eigenvectors

A nonzero vector v is an eigenvector of A with eigenvalue lambda if Av = lambda v. The vector gets scaled but not rotated. Eigenvalues are the roots of the characteristic polynomial det(A - lambda I) = 0. Each eigenvalue has an eigenspace: the set of all eigenvectors for that eigenvalue, plus zero.

Scheme

; Eigenvectors: Av = lambda * v.
; A = ((2 1) (0 3)). Find eigenvalues.
; det(A - lambda*I) = (2-lambda)(3-lambda) - 0 = 0
; lambda = 2 or lambda = 3.

(define (dot u v) (apply + (map * u v)))
(define (mat-vec A v) (map (lambda (row) (dot row v)) A))
(define (scalar-mult c v) (map (lambda (x) (* c x)) v))

(define A '((2 1) (0 3)))

; Eigenvalue lambda=2: solve (A - 2I)v = 0
; ((0 1)(0 1))v = 0 => v2 = 0. Eigenvector: (1, 0).
(define v1 '(1 0))
(display "A * v1 = ") (display (mat-vec A v1)) (newline)
(display "2 * v1 = ") (display (scalar-mult 2 v1)) (newline)

; Eigenvalue lambda=3: solve (A - 3I)v = 0
; ((-1 1)(0 0))v = 0 => v1 = v2. Eigenvector: (1, 1).
(define v2 '(1 1))
(display "A * v2 = ") (display (mat-vec A v2)) (newline)
(display "3 * v2 = ") (display (scalar-mult 3 v2))

Characteristic polynomial

The characteristic polynomial of an n-by-n matrix A is det(A - lambda I), a polynomial of degree n in lambda. Its roots are the eigenvalues. For a 2-by-2 matrix, this is lambda^2 - trace(A)*lambda + det(A) = 0. The trace equals the sum of eigenvalues, and the determinant equals their product.

Scheme

; Characteristic polynomial of a 2x2 matrix.
; A = ((a b)(c d))
; char poly: lambda^2 - (a+d)*lambda + (ad - bc) = 0

(define (char-poly-2x2 a b c d)
  ; Returns (list p q) where poly is lambda^2 + p*lambda + q
  (list (- (+ a d)) (- (* a d) (* b c))))

; A = ((4 2)(1 3))
(define coeffs (char-poly-2x2 4 2 1 3))
(display "lambda^2 + ")
(display (car coeffs))
(display "*lambda + ")
(display (cadr coeffs))
(display " = 0")
(newline)
; lambda^2 - 7*lambda + 10 = 0
; (lambda - 2)(lambda - 5) = 0
; Eigenvalues: 2 and 5

(display "Eigenvalues: 2 and 5")
(newline)
(display "Trace = 4 + 3 = 7 = 2 + 5")
(newline)
(display "Det = 4*3 - 2*1 = 10 = 2 * 5")

Diagonalization

If an n-by-n matrix has n linearly independent eigenvectors, it is diagonalizable: A = P D P^(-1), where D is the diagonal matrix of eigenvalues and P is the matrix whose columns are eigenvectors. In the eigenvector basis, the map is just scaling along each axis. Not all matrices are diagonalizable; those with repeated eigenvalues may fail.

Scheme

; Diagonalization: A = P D P^(-1)
; A = ((4 2)(1 3)), eigenvalues 2 and 5 (from above).
;
; lambda=2: (A-2I)v=0 => ((2,2)(1,1))v=0 => v=(-1,1)
; lambda=5: (A-5I)v=0 => ((-1,2)(1,-2))v=0 => v=(2,1)
;
; P = ((-1 2)(1 1)), D = ((2 0)(0 5))

(define (dot u v) (apply + (map * u v)))
(define (mat-vec A v) (map (lambda (row) (dot row v)) A))

(define A '((4 2) (1 3)))

; Verify by applying A to each eigenvector
(define v1 '(-1 1))
(define v2 '(2 1))

(display "A*(-1,1) = ") (display (mat-vec A v1))
(display "  should be 2*(-1,1) = (-2,2)")
(newline)

(display "A*(2,1)  = ") (display (mat-vec A v2))
(display "  should be 5*(2,1) = (10,5)")
(newline)

; In the eigenvector basis, A becomes diagonal:
;   D = ((2 0) (0 5))
; Powers are easy: D^n = ((2^n 0) (0 5^n))
(display "D^3 = ((8 0)(0 125))")

When diagonalization fails

A matrix with a repeated eigenvalue may not have enough independent eigenvectors to diagonalize. The matrix ((2 1)(0 2)) has eigenvalue 2 with algebraic multiplicity 2 but geometric multiplicity 1 (only one independent eigenvector). Hefferon covers Jordan normal form as the next-best thing: block-diagonal with 1s on the superdiagonal of deficient blocks.

Scheme

; A = ((2 1)(0 2)). Eigenvalue: 2 (repeated).
; (A - 2I) = ((0 1)(0 0)). Kernel = span of (1,0).
; Only one eigenvector. Not diagonalizable.

(define (dot u v) (apply + (map * u v)))
(define (mat-vec A v) (map (lambda (row) (dot row v)) A))
(define (scalar-mult c v) (map (lambda (x) (* c x)) v))

(define A '((2 1) (0 2)))
(define v '(1 0))

(display "A*(1,0) = ") (display (mat-vec A v))
(display " = 2*(1,0)? ") (display (equal? (mat-vec A v) (scalar-mult 2 v)))
(newline)

; Try (0,1): A*(0,1) = (1,2) which is not 2*(0,1) = (0,2).
(define w '(0 1))
(display "A*(0,1) = ") (display (mat-vec A w))
(display " = 2*(0,1)? ") (display (equal? (mat-vec A w) (scalar-mult 2 w)))
(newline)

; Only one eigenvector direction. Not diagonalizable.
; Jordan form: ((2 1)(0 2)) — already in Jordan form.
(display "Jordan form: ((2 1)(0 2))")

Notation reference

Symbol	Scheme	Python	Meaning
Av = lambda v	(mat-vec A v)	A @ v == lam * v	Eigenvalue equation
det(A - lambda I) = 0	(char-poly ...)	np.linalg.eigvals	Characteristic polynomial
A = PDP^(-1)	change of basis	P @ D @ inv(P)	Diagonalization
tr(A)	sum of diagonal	np.trace(A)	Sum of eigenvalues
Jordan form	block diagonal	scipy jordan_form	Best form when not diagonalizable

Neighbors

Adjacent chapters

Ch.4 Determinants — eigenvalues come from det(A - lambda I) = 0
🤖 ML Ch.6 — PCA is eigendecomposition of the covariance matrix
🎛 Control Ch.7 — eigenvalues determine stability in state space
△ Geometry Ch.3 — linear transformations as geometric operations

Cross-references

Sato 2023 — enriched categories use eigenvalue-like structure in their hom-objects
Milewski Ch.1 — similar matrices represent the same morphism in different bases

Foundations (Wikipedia)

Translation notes

Hefferon's final chapter also covers complex eigenvalues, the Cayley-Hamilton theorem, and Jordan normal form in detail. This page covers the core arc from eigenvalues through diagonalization. For the full treatment of similarity and canonical forms, see the original text.

Full chapter: Hefferon, Chapter Five — Similarity.

← Determinants by june.kim fin · 5 of 5