Singular Value Decomposition

The singular value decomposition is useful when the matrix is singular or nearly singular, or when the system of equations is overdetermined. An m by n matrix A can be represented by

where the matrices U and V are orthogonal in the following sense.

In addition

This decomposition can always be done, even for singular matrices. The condition number is the ratio of the largest wj to the smallest wj. The inverse of A is

The rank, r, of a matrix is a value such that all r+1 by r+1 determinants are zero. If an n by n matrix A is singular then the rank of the matrix is r < n. The columns of U whose same-numbered elements wj are nonzero are an orthonormal set of basis vectors that span the range. The columns of V whose same-numbered wj are zero provide an orthonormal basis for the nullspace. The solution to the problem A x = f is

In this equation if wj = 0 we replace 1/wj with zero! [Press, et al., 1986]. This is also the least-squares solution to a set of over-determined equations (i.e. A is an m x n matrix, x is a n by 1 vector, and f is a m x 1 vector).