# Least Square Error Estimation: Conditions for $(A^TA)^{ -1} = A^{-1}(A^T)^{-1}$?

I am new to linear algebra and have this simple question... in least sqaure estimation...the best estimation of the equation $Ax = b$ is $x_{Estimated} = A (A^t A )^{-1} A^t b$...the projection of $b$ on the column space of $A$ is $p = A (A^t A)^{-1} At b$ which is written as $p = P b...P$ is the projection matrix = $A (A^t A)^{-1} A^t$ now using inverse rule $(AB)^{-1} = B^{-1} A^{-1}$ .......$P$ reduces to $I$ ....so is the projection matrix $P$ just the identity matrix $I$ ..?? am i missing out some basic or is my concept not clear? can we write $(A^t A)^{-1} = A^{-1} (A^t)^{-1}$ always??

• thanx Peter for editing...i was missing the superscripts..kind of new to editing with symbols...makes it look proper now...thanx – rotating_image Sep 28 '12 at 12:53
• Also note that least-squares solution of $Ax=b$ can be written as $x=A^{+}b$ where $A^{+}$ is a generalized inverse, or pseudo-inverse. Of course $A^{-1}$ is not defined when $A$ is over-determined. To make $A$ square, the problem is sometimes solved using normal equations of the form $A^{\mathbb{T}}Ax=A^{\mathbb{T}}b$, but in near-singular cases, solving by QR decomposition or SVD is more appropriate. SVD works also for rank-deficient cases. – Libor Sep 28 '12 at 13:42
• do you mean that if the rank of A (say r) r = n(dimension of x) then the projection matrix P = I ? – rotating_image Sep 28 '12 at 13:50
• If $A$ has same rank as dimension of $x$ and $A$ is invertible, then yes. – Libor Sep 28 '12 at 14:09

First of all, the minimum norm least square solution is $A^+b$, where $A^+$ is the pseudoinverse. Only when the left inverse $A_L^{-1}$ (or right inverse $A_R^{-1}$) exists, you have $A^+=A_L^{-1}=(A^TA)^{-1}A^T$ (or $A^+=A_R^{-1}=A^T(AA^T)^{-1}$).
Likewise, the projection matrix onto the column space is $P=AA^+$, or $P=AA_R^{-1}=AA^T(AA^T)^{-1}=I$ if $A_R^{-1}$ exists. In other words, if A has full row rank, then the projector $P$ is indeed an identity matrix. If only left inverse exists, then $P=AA_L^{-1}=A(A^TA)^{-1}A^T$ is not an identity matrix.
Finally, $(AB)^{-1}=A^{-1}B^{-1}$ is valid only when $A$ and $B$ are square matrices.
$$(AB)(B^{-1}A^{-1}) = A (BB^{-1}) A^{-1} = A A^{-1} = I$$
So $(AB)^{-1} = B^{-1} A^{-1}$ provided $A$ and $B$ are invertible.