The main idea of many of the methods I've already mentioned and the one we've yet to cover in this tutorial is to decompose a matrix into a product of a few matrices with special properties, and then exploit these properties to get what we want. Not all decompositions, aka factorizations, are the same; some give more benefits than others. On the other hand, they may be more difficult to compute, or require yet another set of properties from the original matrix. Recall that for random matrices, we have to do LU decomposition with pivoting, but for the positive definite, pivoting is not needed, while for thex triangular, no decomposition is needed.

As diagonal matrices are very convenient, and computationally efficient, it makes sense to have a method to decompose a matrix into some diagonal form. One such decomposition, a very powerful one, is the Singular Value Decomposition (SVD). It is relatively expensive to compute, but once we have it, it can be used to get answers for some difficult cases when other decompositions can not help us.