Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix into a rotation, followed by a rescaling followed by another rotation. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any ${\displaystyle m\times n}$ matrix. It is related to the polar decomposition.

Specifically, the singular value decomposition of an ${\displaystyle m\times n}$ complex matrix ${\displaystyle \mathbf {M} }$ is a factorization of the form ${\displaystyle \mathbf {M} =\mathbf {U\Sigma V^{*}} ,}$ where ${\displaystyle \mathbf {U} }$ is an ${\displaystyle m\times m}$ complex unitary matrix, ${\displaystyle \mathbf {\Sigma } }$ is an ${\displaystyle m\times n}$ rectangular diagonal matrix with non-negative real numbers on the diagonal, ${\displaystyle \mathbf {V} }$ is an ${\displaystyle n\times n}$ complex unitary matrix, and ${\displaystyle \mathbf {V} ^{*}}$ is the conjugate transpose of ${\displaystyle \mathbf {V} }$. Such decomposition always exists for any complex matrix. If ${\displaystyle \mathbf {M} }$ is real, then ${\displaystyle \mathbf {U} }$ and ${\displaystyle \mathbf {V} }$ can be guaranteed to be real orthogonal matrices; in such contexts, the SVD is often denoted ${\displaystyle \mathbf {U} \mathbf {\Sigma } \mathbf {V} ^{\mathrm {T} }.}$

The diagonal entries ${\displaystyle \sigma _{i}=\Sigma _{ii}}$ of ${\displaystyle \mathbf {\Sigma } }$ are uniquely determined by ${\displaystyle \mathbf {M} }$ and are known as the singular values of ${\displaystyle \mathbf {M} }$. The number of non-zero singular values is equal to the rank of ${\displaystyle \mathbf {M} }$. The columns of ${\displaystyle \mathbf {U} }$ and the columns of ${\displaystyle \mathbf {V} }$ are called left-singular vectors and right-singular vectors of ${\displaystyle \mathbf {M} }$, respectively. They form two sets of orthonormal bases ${\displaystyle \mathbf {u} _{1},\ldots ,\mathbf {u} _{m}}$ and ${\displaystyle \mathbf {v} _{1},\ldots ,\mathbf {v} _{n},}$ and if they are sorted so that the singular values ${\displaystyle \sigma _{i}}$ with value zero are all in the highest-numbered columns (or rows), the singular value decomposition can be written as

$${\displaystyle \mathbf {M} =\sum _{i=1}^{r}\sigma _{i}\mathbf {u} _{i}\mathbf {v} _{i}^{*},}$$

where ${\displaystyle r\leq \min\{m,n\}}$ is the rank of ${\displaystyle \mathbf {M} .}$

The SVD is not unique, however it is always possible to choose the decomposition such that the singular values ${\displaystyle \Sigma _{ii}}$ are in descending order. In this case, ${\displaystyle \mathbf {\Sigma } }$ (but not ${\displaystyle \mathbf {U} }$ and ${\displaystyle \mathbf {V} }$) is uniquely determined by ${\displaystyle \mathbf {M} .}$

The term sometimes refers to the compact SVD, a similar decomposition ${\displaystyle \mathbf {M} =\mathbf {U\Sigma V} ^{*}}$ in which ${\displaystyle \mathbf {\Sigma } }$ is square diagonal of size ${\displaystyle r\times r,}$ where ${\displaystyle r\leq \min\{m,n\}}$ is the rank of ${\displaystyle \mathbf {M} ,}$ and has only the non-zero singular values. In this variant, ${\displaystyle \mathbf {U} }$ is an ${\displaystyle m\times r}$ semi-unitary matrix and ${\displaystyle \mathbf {V} }$ is an ${\displaystyle >n\times r}$ semi-unitary matrix, such that ${\displaystyle \mathbf {U} ^{*}\mathbf {U} =\mathbf {V} ^{*}\mathbf {V} =\mathbf {I} _{r}.}$

Mathematical applications of the SVD include computing the pseudoinverse, matrix approximation, and determining the rank, range, and null space of a matrix. The SVD is also extremely useful in all areas of science, engineering, and statistics, such as signal processing, least squares fitting of data, and process control.