What is benefit to use SVD for solving Ax=b

Question

I have a linear equation such as

Ax=b

where A is full rank matrix which its size is 512x512. b is a vector of 512x1. x is unknown vector. I want to find x, hence, I have some options for doing that

1.Using the normal way

inv(A)*b

2.Using SVD ( Singular value decomposition)

[U S V]=svd(A);

x = V*(diag(diag(S).^-1)*(U.'*b))

Both methods give the same result. So, what is benefit of using SVD to solve Ax=b, especially in the case A is a 2D matrix?

Answer 1

inv(A)*b has several negative sides. The main one is that it explicitly calculates the inverse of A, which is both time demanding, and may result in inaccuracies if values vary by many orders of magnitude.

Although it might be better than inv(A)*b, using svd is not the "correct" approach here. The MATLAB-way to do this is using mldivide, \. Using this, MATLAB chooses the best algorithm to solve the linear system based on its properties (Hermation, upper Hessenberg, real and positive diagonal, symmetric, diagonal, sparse etc.). Often, the solution will be a LU-triangulation with partial permutation, but it varies. You'll have a hard time beating MATLABs implementation of mldivide, but using svd might give you some more insight of the properties of the system if you actually investigates U, S, V. If you don't want to do that, do with mldivide.

Answer 2

Welcome to the world of numerical methods, let me be your guide.

You, as a new person in this world wonders, "Why would I do something this difficult with this SVD stuff instead of the so commonly known inverse?! Im going to try it in Matlab!"

And no answer was found. That is, because you are not looking at the problem itself! The problems arise when you have an ill-conditioned matrix. Then the computing of the inverse is not possible numerically.

example:

A=[1    1  -1;
   1   -2   3;
   2   -1   2];

try to invert this matrix using inv(A). Youll get infinite. That is, because the condition number of the matrix is very high (cond(A)).

However, if you try to solve it using SVD method (b=[1;-2;3]) you will get a result. This is still a hot research topic. Solving Ax=b systems with ill condition numbers.

As @Stewie Griffin suggested, the best way to go is mldivide, as it does a couple of things behind it.

(yeah, my example is not very good because the only solution of X is INF, but there is a way better example in this youtube video)