11
$\begingroup$

The classical definition of orthogonality in linear algebra is that two vectors are orthogonal, if their inner product is zero.

I thought this definition might be applied to signals as well, but then I thought about the following example:

Consider a signal in the form of a sine-wave, and another signal in the form of a cosine-wave. If I sample both of them, I obtain two vectors. While sine and cosine are orthogonal functions, the product of the sampled vectors is almost never zero, nor does their cross-correlation function at t=0 vanish.

So how, then, is orthogonality defined in this case? Or is my example off?

Improve this question
$\endgroup$
11
$\begingroup$

As you may know, orthogonality depends on the inner product of your vector space. In your question you state that:

While sine and cosine are orthogonal functions...

This means that you have probably heard of the "standard" inner product for function spaces:

$$\langle f,g\rangle=\int\limits_{x_1}^{x_2} f(x)g(x) \ \mathrm{d}x$$

If you solve this integral for $f(x)=\cos (x)$ and $g(x)=\sin(x)$ for a single period, the result will be $0$: they are orthogonal.

Sampling these signals, however, is not related to orthogonality or anything. The "vectors" you obtain when you sample a signal are just values put together that make sense to you: they are not strictly vectors, they are just arrays (in programming slang). The fact that we call them vectors in MATLAB or any other programming language can be confusing.

It's a bit tricky, actually, since one could define a vector space of dimension $N$ if you have $N$ samples for each signal, where those arrays would indeed be actual vectors. But those would define different things.

For simplicity, let's suppose we are in vector space $\mathbb{R}^3$ and you have $3$ samples for each signal, and all of them are real-valued. In the first case, a vector (i.e. three numbers put together) would refer to a position in space. In the second one, they refer to three values a signal reaches at three different times. In this example it is easy to spot the difference. If you had $n$ samples, then the notion of "space" would be less intuitive, but the idea still holds.

In a nutshell, two signals are orthogonal if the inner product between them (namely, the integral I wrote above) is $0$, and the vectors/arrays obtained by sampling them tell us nothing about their being orthogonal.

Improve this answer
$\endgroup$
7
  • 4
    $\begingroup$ The term "vector" does not necessarily mean "a position in space". In fact, any element from a vector space can be considered a vector. The function space L2 is also a vector space with element wise addition and scalar multiplication. Hence, functions that are element of L2 can considered vectors of this vector space. As such, the inner product between these vectors determines, if the functions are orthogonal on this vector space. $\endgroup$ – Maximilian Matthé Feb 2 '17 at 13:16
  • $\begingroup$ Hi @MaximilianMatthé, I never stated that "vector" = "position in space". I wrote the example of the vector space $\mathbb{R}^3$ to make things clearer, and in that case vector are in general space coordinates. The fact that I defined an inner product for functions states (implicitly) that functions can form a vector space. Should I edit anything in my post to make it clearer? I was referring to samples not composing the same vector space as the signals themselves, and that's the reason why the samples don't say anything about orthogonality. $\endgroup$ – Tendero Feb 2 '17 at 13:21
  • $\begingroup$ @Tendero Thank you (I asked the question, forgot to log in before)! However, I am still struggling, because you stated that, if I calculated the given integral with $f(x)=cos(x)$ and $g(x)=sin(x)$, then I would get $0$. Well, no. The result is $-0.5cos^2(x)$, which is not always zero. Granted, if I integrate over one period, then I get zero. But in reality I have non-periodic functions to start with, and their inner product (as defined by your integral) isn't periodic, either. So what then? $\endgroup$ – AlphaOmega Feb 2 '17 at 13:30
  • 2
    $\begingroup$ @AlphaOmega Functions are orthogonal in determined intervals. The integration interval must be defined in order to know if two functions are orthogonal in that interval. The usual thing is to integrate the cosine and sine in a period, and then the inner product is $0$. If you have non-periodic functions, then maybe you should ask another question with that stated and see what's up in that case. $\endgroup$ – Tendero Feb 2 '17 at 13:39
  • $\begingroup$ The inner product should always include the boundaries, otherwise the inner product isn't a function to a field. Which interval you choose also alters the vector space one talks about. $\endgroup$ – syntonym Feb 2 '17 at 17:04
6
$\begingroup$

Orthogonality is indeed defined via an inner product, with an integral for a continuous ordinal time variable, with a sum for a discrete time variable.

When you convert two (continuous) orthogonal signals into discrete ones (regular sampling, discrete amplitudes), possibly windowed (finite support), you can affect the orthogonality. In other words: two orthogonal continuous-time signals can become only near-orthogonal when discretized. If the discretization is fine enough, and the window well-chosen, then in some cases (pertaining to periodicity, frequency), you maintain orthogonality.

In the continuous setting, the function space is infinite, so you have a lot of options to find orthogonal signals. In a discrete space, the maximum number of mutually orthogonal signals is limited by the dimension of the space.

Improve this answer
$\endgroup$
2
$\begingroup$

You first have to define an inner product for functions. You can't just multiply with each other.

I am not sure about the properties of inner product myself, but according to this lecture an inner product has to be commutative, linear and the inner product of a function with itself should be positive definite.

One option for an inner product for functions could be,

$$ \left\langle f_1, f_2 \right\rangle = \int_a^b f_1(x)\, f_2(x)\, \mathrm{d}x, $$

with $a<b$. But maybe you could come up with different definitions yourself, or play with this one and see for which $a$ and $b$, $\sin(x)$ and $\cos(x)$ are orthogonal.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Actually, $sin(2\pi k_1 f_0 t)$ and $\cos(2\pi k_2 f_0 t)$ are orthogonal for $b-a=\frac{n}{f_0}$ and $k_1,k_2\in\mathbb{Z}$ with $n\in\mathbb{Z}$. Thats the fundamental period of both functions. $\endgroup$ – Maximilian Matthé Feb 2 '17 at 13:20
  • 1
    $\begingroup$ Inner products aren't linear -- they're bilinear for real vector spaces and sesquilinear for complex ones. They're symmetric for real vector spaces and conjugate symmetric for complex ones. $\endgroup$ – Batman Feb 2 '17 at 17:10
2
$\begingroup$

I think I can answer the question after reading the article "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis" by Huang. In this paper (Page 927), Huang gave the defination of the orthogonality between two signals: The defination of orthogonality between signals

And also, I'd like to share with you my MATLAB code:

function OC=ort(x,y)
x=x(:)';
y=y(:);
xy=x*y;
OC=xy/(sum(x.^2)+sum(y.^2));
end

That's all, Good luck~

Improve this answer
$\endgroup$
1
$\begingroup$

In terms of matrix multiplication (such as for a DFT), the equivalent interval of integration for signals is determined by the size of the matrix (or the size of the input vector) and the sample rate. These are often chosen due to practical considerations (time or space of interest and/or of availability, etc.). Orthogonality is defined over that interval of integration.

Improve this answer
$\endgroup$
0
$\begingroup$

I would argue that your example is a bit off.

You very likely didn't sample the functions $\sin$ and $\cos$ properly, in the sense that the sampling should respect their periodicity. If you sample these function on the set $\{\dfrac{n2\pi}{N}\ |\ n\in\{0,\ldots, N-1\}\}$, I assure you that you'll find that the $N$-dimensional vectors you'll find will be fully orthogonal.

Improve this answer
$\endgroup$
0
$\begingroup$

I like to have a geometrical approach on this type of problem by remembering that Pythogoras formula still holds for vectors:

$$ | x - y |^2 = | x |^2 + | y |^2 - 2\cdot\left\langle x, y \right\rangle, $$

with the scalar product defining the correlation coefficient as the cosinus of the angle between the two vectors in this inner product space :

$$ \left\langle x, y \right\rangle = | x | \cdot | y | \cdot \cos( angle(x, y)), $$

The scalar $\cos(angle(x, y))$ is thus bounded between $-1$ and $1$ and measures the cosine of the angle $angle(x, y)$ between the vectors $x$ and $y$.

Geometric interpretation of the angle between two vectors defined using an inner product

such that, to answer your question, orthogonality is defined (as in the planar space of usual geometry) as when the cosine is zero.

Improve this answer
$\endgroup$
4
  • $\begingroup$ what do you mean by $\cos(f,g)$? $\endgroup$ – robert bristow-johnson Feb 3 '17 at 9:19
  • $\begingroup$ $\cos$ is the scalar defined by the second equation, I added a ink + tried to make that clearer $\endgroup$ – meduz Feb 4 '17 at 11:57
  • $\begingroup$ you mean: $$\begin{align} \cos(f,g) & \triangleq \frac{\langle f, g \rangle}{|f| \cdot |g|} \\ \\ &= \frac{|f|^2+|g|^2-|f-g|^2}{2 \cdot |f| \cdot |g|} \\ \end{align}$$ is that what you're saying? i have never seen a two-argument cosine function in the near half century that i had been aware of a cosine function. $\endgroup$ – robert bristow-johnson Feb 4 '17 at 20:27
  • $\begingroup$ you are right, my mistake, I have corrected the formulation of my answer. $\endgroup$ – meduz Feb 19 '17 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.