Vector Spaces

Jim Hefferon · GFDL + CC BY-SA 2.5 · Linear Algebra

A vector space is a set with addition and scalar multiplication that obey ten axioms. Subspaces, linear independence, span, basis, and dimension all follow from this definition. Every vector space has a basis, and every basis has the same number of elements.

Definition and examples

A vector space over a field F is a set V with two operations: vector addition and scalar multiplication. The ten axioms guarantee closure, associativity, commutativity of addition, existence of zero and additive inverses, and compatibility of scalar multiplication. The real numbers R^n, the set of polynomials of degree at most n, and the set of m-by-n matrices are all vector spaces.

Scheme

; A vector in R^n is a list of numbers.
; Vector addition and scalar multiplication.

(define (vec-add v w)
  (map + v w))

(define (scalar-mult c v)
  (map (lambda (x) (* c x)) v))

; Examples in R^3
(define v '(1 2 3))
(define w '(4 5 6))

(display "v + w = ")
(display (vec-add v w))
(newline)

(display "3 * v = ")
(display (scalar-mult 3 v))
(newline)

; Zero vector
(define zero '(0 0 0))
(display "v + 0 = ")
(display (vec-add v zero))
(newline)

; Additive inverse
(define neg-v (scalar-mult -1 v))
(display "v + (-v) = ")
(display (vec-add v neg-v))

Linear independence

Vectors v1, ..., vn are linearly independent if the only solution to c1*v1 + ... + cn*vn = 0 is c1 = ... = cn = 0. If there is a nontrivial solution, the vectors are dependent: at least one can be written as a combination of the others. Independence means no redundancy.

Scheme

; Check linear independence by row reducing.
; Columns are the vectors; if every column has a pivot, independent.

; Are (1,2) and (3,4) independent?
; Form matrix with these as columns, then check.
; Equivalently: does c1*(1,2) + c2*(3,4) = (0,0) force c1=c2=0?
;   c1 + 3*c2 = 0
;   2*c1 + 4*c2 = 0
; R2 <- R2 - 2*R1: (0, -2, 0)
; Two pivots => independent.

(define (vec-add v w) (map + v w))
(define (scalar-mult c v) (map (lambda (x) (* c x)) v))

; Test: is c1*v1 + c2*v2 = 0 only when c1=c2=0?
(define v1 '(1 2))
(define v2 '(3 4))

; Try c1=4, c2=-2: should NOT give zero if independent
(define combo (vec-add (scalar-mult 4 v1) (scalar-mult -2 v2)))
(display "4*v1 - 2*v2 = ")
(display combo)
(newline)

; Now try dependent vectors: v3 = 2*v1
(define v3 '(2 4))
(display "v1 and 2*v1 are dependent: ")
(display (vec-add v1 (scalar-mult -1/2 v3)))
; = (0 0) with coefficients 1 and -1/2

Basis and dimension

A basis for a vector space V is a linearly independent set that spans V. Every element of V can be written as a unique linear combination of basis vectors. The number of vectors in any basis is always the same: that number is the dimension of V.

Scheme

; The standard basis for R^3: e1, e2, e3.
; Any vector (a,b,c) = a*e1 + b*e2 + c*e3.

(define e1 '(1 0 0))
(define e2 '(0 1 0))
(define e3 '(0 0 1))

(define (vec-add v w) (map + v w))
(define (scalar-mult c v) (map (lambda (x) (* c x)) v))

; Express (3, -1, 7) in the standard basis
(define v (vec-add (scalar-mult 3 e1)
           (vec-add (scalar-mult -1 e2)
                    (scalar-mult 7 e3))))
(display "(3,-1,7) = 3*e1 + (-1)*e2 + 7*e3 = ")
(display v)
(newline)

; Dimension of R^3 is 3 (three basis vectors).
; A subspace like the xy-plane has dimension 2.
(display "dim(R^3) = 3")
(newline)
(display "dim(xy-plane in R^3) = 2")

Subspaces

A subspace is a subset of a vector space that is itself a vector space under the same operations. You only need to check: it contains the zero vector, and it is closed under addition and scalar multiplication. Lines and planes through the origin are subspaces of R^3.

Scheme

; The solution set of a homogeneous system is a subspace.
; x + 2y + z = 0 defines a plane through the origin in R^3.

(define (vec-add v w) (map + v w))
(define (scalar-mult c v) (map (lambda (x) (* c x)) v))

; Two solutions of x + 2y + z = 0:
(define s1 '(-2 1 0))   ; x = -2y, z = 0
(define s2 '(-1 0 1))   ; x = -z, y = 0

; Check: s1 satisfies the equation
(display "-2 + 2(1) + 0 = ")
(display (+ -2 (* 2 1) 0))
(newline)

; Check closure: s1 + s2 should also satisfy
(define s3 (vec-add s1 s2))
(display "s1 + s2 = ")
(display s3)
(newline)
(display "Check: ")
(display (+ (car s3) (* 2 (cadr s3)) (caddr s3)))
(newline)

; s1 and s2 form a basis for this 2D subspace.
(display "The solution plane has dimension 2.")

Notation reference

Symbol	Scheme	Python	Meaning
v + w	(vec-add v w)	[a+b for a,b in zip(v,w)]	Vector addition
c * v	(scalar-mult c v)	[c*x for x in v]	Scalar multiplication
span(S)	all linear combos	all linear combos	Set of all linear combinations
dim(V)	basis size	len(basis)	Number of basis vectors
R^n	lists of length n	lists of length n	n-dimensional real space

Neighbors

Adjacent chapters

Ch.1 Linear Systems — the computational backbone
Ch.3 Maps Between Spaces — structure-preserving functions between vector spaces
🔗 Abstract Algebra Ch.1 — vector spaces are modules over a field; groups are nearby
🤖 ML Ch.6 — PCA lives in the subspace structure of vector spaces
🎰 Probability Ch.7 — probability distributions form a vector space

Foundations (Wikipedia)

Translation notes

Hefferon proves the dimension theorem (any two bases have the same size) via the Replacement Lemma. The chapter also covers coordinates, change of basis, and the row space / column space distinction. This page focuses on definitions and intuition. For the proofs, see the original text.

Full chapter: Hefferon, Chapter Two — Vector Spaces.

← Linear Systems by june.kim Maps Between Spaces →