The inverse of an orthogonal matrix is its transpose

Question

In the following statement I don't understand the case for $\ i = j$:

Let $\mathbf A$ be an $\ m \times \ n$ orthogonal matrix where $\ a_i$ is the $\ i^{th}$ column vector. The $\ ij^{th} $ element of $\mathbf A^{T}\mathbf A$ is

$$ \left(\mathbf A^T \mathbf A\right)_{ij} = a_i^{T} a_j = \begin{cases} 1 &\text{if} &i = j\\ 0 &\text{if} &\text{otherwise}\\ \end{cases} $$

Therefore, because $\mathbf A^{T} \mathbf A = \mathbf I $, it follows that $\mathbf A^{-1} = \mathbf A^{T}$.

Answer 1

An orthogonal matrix has orthogal columns, i.e. the scalar product of two different columns is zero (the case $i\neq j$). For the case $i=j$ you have $a_i^Ta_i=\|a_i\|^2>0$.

So, all you can say about an orthogonal matrix with colums $a_i$ is:

$$ a_i^Ta_j=\begin{cases}c_i>0 & i=j\\ 0 & i\neq j\end{cases}$$

where $c_i>0$ is some constant, depending on the matrix.

Then, there are orthonormal matrices. These fulfill, in addition to the previous statement, $c_i=1$ and hence $A^TA=I$.

Finally, only if your $A$ is a square orthonormal matrix, it holds $A^{-1}=A^T$.

The task you describe has two flaws:

• it only talks about ortogonal matrices, where we can just say $A^TA=\text{diag}(\vec{c})$ with $\vec{c}$ contains the norms of the columns.
• It states $A$ is an $n\times m$ matrix (i.e. not square). Rectangular matrices dont have an inverse. So, the last statement is not correct.

Edit: Normally in literature, orthogonal matrices are considered to be square with unit-norm columns. Then $A^TA=I$ and $A^{-1}=A^T$. However, as the question explicitely mentions rectangular matrices, I want to point out that it really depends on your definition of what an orthogonal matrix is. And, if you just define it as orthogonal columns, the arguments above hold.

Answer 2

Suppose you have a set $n$ vectors $o_k$ in $\mathbb{R}^{m,1}$ (length $m$, column-style), pairwise orthonormal, i.e. orthogonal with unit norm. Stack them side by side in a matrix $$A = \left[o_1,\ldots,o_n\right]\,.$$

You have what you call an orthogonal (rectangular) matrix, sometimes called an orthogonal column matrix. The same concept applies row-wise. First, note that you should have $m\ge n$. If not, it is not possible to find strictly more than $m$ linearly independent vectors in an $m$ dimensional linear space.

Then, each element $p_{i,j}$ of $A^T A$ is given by $o_i^T o_j$, so $1$ when $i=j$ and $0$ otherwise, by definition of the orthonormality of the $o_k$.

Note that for non-square matrices, there exists so-called generalized inverses, such as the Moore-Penrose pseudo-inverse, that coincides with the standard inverse in the square case..