Show description

For any unitary matrix Q, \\Qx\\2 = iQx,Qx) = (x,Q*Qx) = (x,x) = ||z||2 since Q*Q — I. - zL ' 0 and d\lJ — 0 otherwise. We therefore take y — E'QJfc, and y is the smallest least squares solution. The vector x = Q2VQ\b has the same norm as y and therefore gives the smallest least squares solution of Ax = b. Hence A! — C^E'Q*, as stated earlier. We can illustrate the geometrical meaning of this pseudo-inverse first for symmetric matrices. If A — QAQ” 1, with Q unitary, we know that Q represents the basis relative to which A has a diagonal representation.

5 LEAST SQUARES SOLUTIONS—PSEUDO INVERSES 41 and determines the smallest element whose image under A is Q~l b. 6). The change of coordinates Q identifies for us the range and null space of A. What we have actually done is to project the vector b onto the range of A and then find another vector in the range of A whose image is this projection. Since A = A*, this is the correct thing to do. This construction of A! can work only for symmetric matrices where R(A) = R(A *). 8). First since A ^ A*} R(A) / R(A*) so projecting onto R(A) does not also put us on R(A*).

Suppose xp is one solution of the least squares problem A*Ax = A*6. Since A has linearly dependent columns, there are vectors w that satisfy Aw — 0, and x = xp -{-w is also a least squares solution. One reasonable way to specify the solution uniquely is to seek the smallest possible solution of A* Ax = A*b. As a result, we seek x so that (x, w) = 0 for 30 CHAPTER 1 FINITE DIMENSIONAL VECTOR SPACES all w with Aw = 0. That is, we want x to be orthogonal to the null space of A, and therefore, in the range of A*.