When are two signals orthogonal?

Question

The classical definition of orthogonality in linear algebra is that two vectors are orthogonal, if their inner product is zero.

I thought this definition might be applied to signals as well, but then I thought about the following example:

Consider a signal in the form of a sine-wave, and another signal in the form of a cosine-wave. If I sample both of them, I obtain two vectors. While sine and cosine are orthogonal functions, the product of the sampled vectors is almost never zero, nor does their cross-correlation function at t=0 vanish.

So how, then, is orthogonality defined in this case? Or is my example off?

Answer 1

As you may know, orthogonality depends on the inner product of your vector space. In your question you state that:

While sine and cosine are orthogonal functions...

This means that you have probably heard of the "standard" inner product for function spaces:

$$\langle f,g\rangle=\int\limits_{x_1}^{x_2} f(x)g(x) \ \mathrm{d}x$$

If you solve this integral for $f(x)=\cos (x)$ and $g(x)=\sin(x)$ for a single period, the result will be $0$: they are orthogonal.

Sampling these signals, however, is not related to orthogonality or anything. The "vectors" you obtain when you sample a signal are just values put together that make sense to you: they are not strictly vectors, they are just arrays (in programming slang). The fact that we call them vectors in MATLAB or any other programming language can be confusing.

It's a bit tricky, actually, since one could define a vector space of dimension $N$ if you have $N$ samples for each signal, where those arrays would indeed be actual vectors. But those would define different things.

For simplicity, let's suppose we are in vector space $\mathbb{R}^3$ and you have $3$ samples for each signal, and all of them are real-valued. In the first case, a vector (i.e. three numbers put together) would refer to a position in space. In the second one, they refer to three values a signal reaches at three different times. In this example it is easy to spot the difference. If you had $n$ samples, then the notion of "space" would be less intuitive, but the idea still holds.

In a nutshell, two signals are orthogonal if the inner product between them (namely, the integral I wrote above) is $0$, and the vectors/arrays obtained by sampling them tell us nothing about their being orthogonal.

Answer 2

Orthogonality is indeed defined via an inner product, with an integral for a continuous ordinal time variable, with a sum for a discrete time variable.

When you convert two (continuous) orthogonal signals into discrete ones (regular sampling, discrete amplitudes), possibly windowed (finite support), you can affect the orthogonality. In other words: two orthogonal continuous-time signals can become only near-orthogonal when discretized. If the discretization is fine enough, and the window well-chosen, then in some cases (pertaining to periodicity, frequency), you maintain orthogonality.

In the continuous setting, the function space is infinite, so you have a lot of options to find orthogonal signals. In a discrete space, the maximum number of mutually orthogonal signals is limited by the dimension of the space.

Answer 3

You first have to define an inner product for functions. You can't just multiply with each other.

I am not sure about the properties of inner product myself, but according to this lecture an inner product has to be commutative, linear and the inner product of a function with itself should be positive definite.

One option for an inner product for functions could be,

$$ \left\langle f_1, f_2 \right\rangle = \int_a^b f_1(x)\, f_2(x)\, \mathrm{d}x, $$

with $a<b$. But maybe you could come up with different definitions yourself, or play with this one and see for which $a$ and $b$, $\sin(x)$ and $\cos(x)$ are orthogonal.

Answer 4

I think I can answer the question after reading the article "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis" by Huang. In this paper (Page 927), Huang gave the defination of the orthogonality between two signals:

And also, I'd like to share with you my MATLAB code:

function OC=ort(x,y)
x=x(:)';
y=y(:);
xy=x*y;
OC=xy/(sum(x.^2)+sum(y.^2));
end

That's all, Good luck~

Answer 5

In terms of matrix multiplication (such as for a DFT), the equivalent interval of integration for signals is determined by the size of the matrix (or the size of the input vector) and the sample rate. These are often chosen due to practical considerations (time or space of interest and/or of availability, etc.). Orthogonality is defined over that interval of integration.

Answer 6

I would argue that your example is a bit off.

You very likely didn't sample the functions $\sin$ and $\cos$ properly, in the sense that the sampling should respect their periodicity. If you sample these function on the set $\{\dfrac{n2\pi}{N}\ |\ n\in\{0,\ldots, N-1\}\}$, I assure you that you'll find that the $N$-dimensional vectors you'll find will be fully orthogonal.

Answer 7

I like to have a geometrical approach on this type of problem by remembering that Pythogoras formula still holds for vectors:

$$ | x - y |^2 = | x |^2 + | y |^2 - 2\cdot\left\langle x, y \right\rangle, $$

with the scalar product defining the correlation coefficient as the cosinus of the angle between the two vectors in this inner product space :

$$ \left\langle x, y \right\rangle = | x | \cdot | y | \cdot \cos( angle(x, y)), $$

The scalar $\cos(angle(x, y))$ is thus bounded between $-1$ and $1$ and measures the cosine of the angle $angle(x, y)$ between the vectors $x$ and $y$.

such that, to answer your question, orthogonality is defined (as in the planar space of usual geometry) as when the cosine is zero.