You are here

Linear Algebra and Matrix Analysis for Statistics

Sudipto Banerjee and Anindya Roy
Chapman & Hall/CRC
Publication Date: 
Number of Pages: 
Texts in Statistical Science
[Reviewed by
Robert W. Hayden
, on

At last — a book whose content is broader than its title! This would be a reasonable candidate for use in a standard linear algebra course, even at institutions with no statistics majors. The word “statistics” in the title only indicates that preference has been given to topics used in statistics. Just how they are used receives scant attention, and students (and many teachers) using the text might well be unaware it has any special orientation toward statistics.

The presentation is pretty much straight theorem-proof. The proofs are very detailed and the authors bind the argument together with clear text that flows beautifully. Asides here and there address what disciplines find the current topic useful, or provide valuable background information. The book is also oriented toward applications in that it often discusses the pros and cons of various computational approaches, though students do little computation by hand, and no software support is offered.

The authors strive to find a middle ground between the geometric approach of classic texts such as Halmos and the more recent approach via matrix forms made popular by Strang. The vector spaces are usually over the reals. The two principal exceptions are complex vector spaces for eigenvalues, and a final chapter on abstract vector spaces. The exercises are varied if not numerous. Most ask the student to prove some minor result. Some ask for counterexamples. A few have a student do a computation just to make the theory concrete. It is doubtful that many students would experience any of these as “applications.” One might describe this as a textbook in applicable linear algebra.

Some linear algebra courses put a greater emphasis on concrete applications or on using software to get computations done. Other texts treat linear algebra as a branch of abstract algebra and allow spaces over arbitrary fields. This book is a strong contender for the vast majority of linear algebra courses that fall between those two extremes.

After a few years in industry, Robert W. Hayden ( taught mathematics at colleges and universities for 32 years and statistics for 20 years. In 2005 he retired from full-time classroom work. He now teaches statistics online at and does summer workshops for high school teachers of Advanced Placement Statistics. He contributed the chapter on evaluating introductory statistics textbooks to the MAA's Teaching Statistics.

Matrices, Vectors, and Their Operations
Basic definitions and notations
Matrix addition and scalar-matrix multiplication
Matrix multiplication
Partitioned matrices
The "trace" of a square matrix
Some special matrices

Systems of Linear Equations
Gaussian elimination
Gauss-Jordan elimination
Elementary matrices
Homogeneous linear systems
The inverse of a matrix

More on Linear Equations
The LU decomposition
Crout’s Algorithm
LU decomposition with row interchanges
The LDU and Cholesky factorizations
Inverse of partitioned matrices
The LDU decomposition for partitioned matrices
The Sherman-Woodbury-Morrison formula

Euclidean Spaces
Vector addition and scalar multiplication
Linear spaces and subspaces
Intersection and sum of subspaces
Linear combinations and spans
Four fundamental subspaces
Linear independence
Basis and dimension

The Rank of a Matrix
Rank and nullity of a matrix
Bases for the four fundamental subspaces
Rank and inverse
Rank factorization
The rank-normal form
Rank of a partitioned matrix
Bases for the fundamental subspaces using the rank normal form

Complementary Subspaces
Sum of subspaces
The dimension of the sum of subspaces
Direct sums and complements

Orthogonality, Orthogonal Subspaces, and Projections
Inner product, norms, and orthogonality
Row rank = column rank: A proof using orthogonality
Orthogonal projections
Gram-Schmidt orthogonalization
Orthocomplementary subspaces
The fundamental theorem of linear algebra

More on Orthogonality
Orthogonal matrices
The QR decomposition
Orthogonal projection and projector
Orthogonal projector: Alternative derivations
Sum of orthogonal projectors
Orthogonal triangularization

Revisiting Linear Equations
Null spaces and the general solution of linear systems
Rank and linear systems
Generalized inverse of a matrix
Generalized inverses and linear systems
The Moore-Penrose inverse

Some basic properties of determinants
Determinant of products
Computing determinants
The determinant of the transpose of a matrix — revisited
Determinants of partitioned matrices
Cofactors and expansion theorems
The minor and the rank of a matrix
The Cauchy-Binet formula
The Laplace expansion

Eigenvalues and Eigenvectors
Characteristic polynomial and its roots
Spectral decomposition of real symmetric matrices
Spectral decomposition of Hermitian and normal matrices
Further results on eigenvalues
Singular value decomposition

Singular Value and Jordan Decompositions
Singular value decomposition (SVD)
The SVD and the four fundamental subspaces
SVD and linear systems
SVD, data compression and principal components
Computing the SVD
The Jordan canonical form
Implications of the Jordan canonical form

Quadratic Forms
Quadratic forms
Matrices in quadratic forms
Positive and nonnegative definite matrices
Congruence and Sylvester’s law of inertia
Nonnegative definite matrices and minors
Extrema of quadratic forms
Simultaneous diagonalization

The Kronecker Product and Related Operations
Bilinear interpolation and the Kronecker product
Basic properties of Kronecker products
Inverses, rank and nonsingularity of Kronecker products
Matrix factorizations for Kronecker products
Eigenvalues and determinant
The vec and commutator operators
Linear systems involving Kronecker products
Sylvester’s equation and the Kronecker sum
The Hadamard product

Linear Iterative Systems, Norms, and Convergence
Linear iterative systems and convergence of matrix powers
Vector norms
Spectral radius and matrix convergence
Matrix norms and the Gerschgorin circles
SVD – revisited
Web page ranking and Markov chains
Iterative algorithms for solving linear equations

Abstract Linear Algebra
General vector spaces
General inner products
Linear transformations, adjoint and rank
The four fundamental subspaces - revisited
Inverses of linear transformations
Linear transformations and matrices
Change of bases, equivalence and similar matrices
Hilbert spaces