3
$\begingroup$

I am new to linear algebra and have this simple question... in least sqaure estimation...the best estimation of the equation $Ax = b$ is $x_{Estimated} = A (A^t A )^{-1} A^t b$...the projection of $b$ on the column space of $A$ is $p = A (A^t A)^{-1} At b$ which is written as $p = P b...P$ is the projection matrix = $A (A^t A)^{-1} A^t$ now using inverse rule $(AB)^{-1} = B^{-1} A^{-1}$ .......$P$ reduces to $I$ ....so is the projection matrix $P$ just the identity matrix $I$ ..?? am i missing out some basic or is my concept not clear? can we write $(A^t A)^{-1} = A^{-1} (A^t)^{-1}$ always??

$\endgroup$
4
  • $\begingroup$ thanx Peter for editing...i was missing the superscripts..kind of new to editing with symbols...makes it look proper now...thanx $\endgroup$ Sep 28, 2012 at 12:53
  • $\begingroup$ Also note that least-squares solution of $Ax=b$ can be written as $x=A^{+}b$ where $A^{+}$ is a generalized inverse, or pseudo-inverse. Of course $A^{-1}$ is not defined when $A$ is over-determined. To make $A$ square, the problem is sometimes solved using normal equations of the form $A^{\mathbb{T}}Ax=A^{\mathbb{T}}b$, but in near-singular cases, solving by QR decomposition or SVD is more appropriate. SVD works also for rank-deficient cases. $\endgroup$
    – Libor
    Sep 28, 2012 at 13:42
  • $\begingroup$ do you mean that if the rank of A (say r) r = n(dimension of x) then the projection matrix P = I ? $\endgroup$ Sep 28, 2012 at 13:50
  • $\begingroup$ If $A$ has same rank as dimension of $x$ and $A$ is invertible, then yes. $\endgroup$
    – Libor
    Sep 28, 2012 at 14:09

2 Answers 2

5
$\begingroup$

First of all, the minimum norm least square solution is $A^+b$, where $A^+$ is the pseudoinverse. Only when the left inverse $A_L^{-1}$ (or right inverse $A_R^{-1}$) exists, you have $A^+=A_L^{-1}=(A^TA)^{-1}A^T$ (or $A^+=A_R^{-1}=A^T(AA^T)^{-1}$).

Likewise, the projection matrix onto the column space is $P=AA^+$, or $P=AA_R^{-1}=AA^T(AA^T)^{-1}=I$ if $A_R^{-1}$ exists. In other words, if A has full row rank, then the projector $P$ is indeed an identity matrix. If only left inverse exists, then $P=AA_L^{-1}=A(A^TA)^{-1}A^T$ is not an identity matrix.

Finally, $(AB)^{-1}=A^{-1}B^{-1}$ is valid only when $A$ and $B$ are square matrices.

$\endgroup$
3
$\begingroup$

From here:

$$ (AB)(B^{-1}A^{-1}) = A (BB^{-1}) A^{-1} = A A^{-1} = I $$

So $(AB)^{-1} = B^{-1} A^{-1}$ provided $A$ and $B$ are invertible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.