When are two signals orthogonal?

Question

The classical definition of orthogonality in linear algebra is that two vectors are orthogonal, if their inner product is zero.

I thought this definition might be applied to signals as well, but then I thought about the following example:

Consider a signal in the form of a sine-wave, and another signal in the form of a cosine-wave. If I sample both of them, I obtain two vectors. While sine and cosine are orthogonal functions, the product of the sampled vectors is almost never zero, nor does their cross-correlation function at t=0 vanish.

So how, then, is orthogonality defined in this case? Or is my example off?

Answer 1

As you may know, orthogonality depends on the inner product of your vector space. In your question you state that:

While sine and cosine are orthogonal functions...

This means that you have probably heard of the "standard" inner product for function spaces:

$$\langle f,g\rangle=\int\limits_{x_1}^{x_2} f(x)g(x) \ \mathrm{d}x$$

If you solve this integral for $f(x)=\cos (x)$ and $g(x)=\sin(x)$ for a single period, the result will be $0$: they are orthogonal.

Sampling these signals, however, is not related to orthogonality or anything. The "vectors" you obtain when you sample a signal are just values put together that make sense to you: they are not strictly vectors, they are just arrays (in programming slang). The fact that we call them vectors in MATLAB or any other programming language can be confusing.

It's a bit tricky, actually, since one could define a vector space of dimension $N$ if you have $N$ samples for each signal, where those arrays would indeed be actual vectors. But those would define different things.

For simplicity, let's suppose we are in vector space $\mathbb{R}^3$ and you have $3$ samples for each signal, and all of them are real-valued. In the first case, a vector (i.e. three numbers put together) would refer to a position in space. In the second one, they refer to three values a signal reaches at three different times. In this example it is easy to spot the difference. If you had $n$ samples, then the notion of "space" would be less intuitive, but the idea still holds.

In a nutshell, two signals are orthogonal if the inner product between them (namely, the integral I wrote above) is $0$, and the vectors/arrays obtained by sampling them tell us nothing about their being orthogonal.

Answer 2

Orthogonality is indeed defined via an inner product, with an integral for a continuous ordinal time variable, with a sum for a discrete time variable.

When you convert two (continuous) orthogonal signals into discrete ones (regular sampling, discrete amplitudes), possibly windowed (finite support), you can affect the orthogonality. In other words: two orthogonal continuous-time signals can become only near-orthogonal when discretized. If the discretization is fine enough, and the window well-chosen, then in some cases (pertaining to periodicity, frequency), you maintain orthogonality.

In the continuous setting, the function space is infinite, so you have a lot of options to find orthogonal signals. In a discrete space, the maximum number of mutually orthogonal signals is limited by the dimension of the space.

Answer 3

You first have to define an inner product for functions. You can't just multiply with each other.

I am not sure about the properties of inner product myself, but according to this lecture an inner product has to be commutative, linear and the inner product of a function with itself should be positive definite.

One option for an inner product for functions could be,

$$ \left\langle f_1, f_2 \right\rangle = \int_a^b f_1(x)\, f_2(x)\, \mathrm{d}x, $$

with $a<b$. But maybe you could come up with different definitions yourself, or play with this one and see for which $a$ and $b$, $\sin(x)$ and $\cos(x)$ are orthogonal.

Answer 4

I think I can answer the question after reading the article "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis" by Huang. In this paper (Page 927), Huang gave the defination of the orthogonality between two signals:

And also, I'd like to share with you my MATLAB code:

function OC=ort(x,y)
x=x(:)';
y=y(:);
xy=x*y;
OC=xy/(sum(x.^2)+sum(y.^2));
end

That's all, Good luck~

Answer 5

In terms of matrix multiplication (such as for a DFT), the equivalent interval of integration for signals is determined by the size of the matrix (or the size of the input vector) and the sample rate. These are often chosen due to practical considerations (time or space of interest and/or of availability, etc.). Orthogonality is defined over that interval of integration.

Answer 6

I would argue that your example is a bit off.

You very likely didn't sample the functions $\sin$ and $\cos$ properly, in the sense that the sampling should respect their periodicity. If you sample these function on the set $\{\dfrac{n2\pi}{N}\ |\ n\in\{0,\ldots, N-1\}\}$, I assure you that you'll find that the $N$-dimensional vectors you'll find will be fully orthogonal.

Answer 7

I like to have a geometrical approach on this type of problem by remembering that Pythogoras formula still holds for vectors:

$$ | x - y |^2 = | x |^2 + | y |^2 - 2\cdot\left\langle x, y \right\rangle, $$

with the scalar product defining the correlation coefficient as the cosinus of the angle between the two vectors in this inner product space :

$$ \left\langle x, y \right\rangle = | x | \cdot | y | \cdot \cos( angle(x, y)), $$

The scalar $\cos(angle(x, y))$ is thus bounded between $-1$ and $1$ and measures the cosine of the angle $angle(x, y)$ between the vectors $x$ and $y$.

such that, to answer your question, orthogonality is defined (as in the planar space of usual geometry) as when the cosine is zero.

Answer 8

Signals orthogonality solves the problem that close-in-frequency RF signal carriers have when intermodulation degrades SNR EbN0 and BER.

Increasing signals power alone worsens the problem, and while reducing signals power also reduces intermodulation, signal range also decreases.

For 2 carriers to be really close in frequency while showing little intermodulation the key parameters are : signal types, carrier frequencies, (detection) integration interval.

Multi-carrier modern OFDM systems like wireless DVB-T DAB 4G 5G and wired ADSL are built upon RF carriers orthogonality.

Carriers that are 'close' in frequency while showing very low intermodulation exploit the following :

integral( sin(mx) * sin(nx) , x ,-pi , pi ) = 0 ; for any [m n] integers

Following, a MATLAB script showing signal orthogonality key points :

clear all;close all;clc  % clean slate

1.- How orthogonal are 2 tones f1 f2=2*f1

A=[1 1]; % tone amplitudes
f0=100;  % [Hz] base tone
N=[1 2];
f1=N(1)*f0 % [Hz]
f1 =    100
f2=N(2)*f0
f2 =    200
f=[f1 f2];
T1=1/f1 % cycle 1st tone
T2=1/f2 % cycle 2nd tone
nw=5 % one-side detection time window width
t1=-nw*pi/(2*pi*f1); % time interval start
t2=nw*pi/(2*pi*f1); % time interval stop

fs=160*max(f);Ts=1/fs;dt=Ts; % sampling
t=[t1:dt:t2]; % time reference
a=2*pi*f'*t; % constant phases
x1=A(1)*sin(a(1,:));
x2=A(2)*sin(a(2,:));

figure(1)
plot(t,x1,t,x2)
axis([-nw*T1 nw*T1 -2 2])
grid on
xlabel('t')
title(['f1=' num2str(f1) 'Hz  f2=' num2str(f2) 'Hz'])

xp=x1.*x2;
abs(trapz(t,xp))

ans = 1.185846126156020e-20

The integral result measures how much orthogonality there is between these 2 signals, e-17 means next-to-nothing in common, therefore a lot of orthogonality.

2.- what happens if f2 not a multiple of f1

N=[1 .760986];
f=[N(1)*f0 N(2)*f0]
f = 1×2
102 ×
   1.000000000000000   0.760986000000000
t1=-nw*pi/(2*pi*f(1));
t2=nw*pi/(2*pi*f(2));
fs=160*max(f);Ts=1/fs;dt=Ts; % sampling
t=[t1:dt:t2];
a=2*pi*f'*t; % constant phases
x1=A(1)*sin(a(1,:));x2=A(2)*sin(a(2,:));
figure(2)
plot(t,x1,t,x2)
axis([-nw*T1 nw*T1 -2 2])
grid on
xlabel('t')
title(['f1=' num2str(f1) 'Hz  f2=' num2str(f2) 'Hz'])

xp=x1.*x2;
abs(trapz(t,xp))

ans = 0.004982727502142

integral shows both signals have a lot more in common than in the 1st run.

Therefore not so orthogonal now.

3.- There are more orthogonal frequencies than just harmonics

f1=f0; % [Hz]
dn1=.001;
n1=[dn1:dn1:5]+1;
f2=n1*f0;
L1=[];  % logging integral result

for k=1:1:numel(n1)
    f=[f1 f2(k)];
    t1=-nw*pi/(2*pi*f(1)); % time interval start
    t2=nw*pi/(2*pi*f(1)); % time interval stop
    fs=160*f(2);Ts=1/fs;dt=Ts; % sampling
    t=[t1:dt:t2]; % time reference
    a=2*pi*f'*t; % constant phases
    x1=A(1)*sin(a(1,:));
    x2=A(2)*sin(a(2,:));
    xp=x1.*x2;
    L1=[L1 abs(trapz(t,xp))];
end
L1=L1/L1(1);
figure(3)
plot(n1,L1)
xlabel('n1')
grid on
title(['integral result f1=' num2str(f1) ' f2 [' num2str(f1*n1(1)) ' ' num2str(f1*n1(end)) ']'])

L2=log10(L1);

figure(5)
plot(n1,L2)
axis([1 6 -4 .1])
xlabel('n1')
grid on
title(['signal orthogonality f1=' num2str(f1) ' f2 [' num2str(f1*n1(1)) ' ' num2str(f1*n1(end)) ']'])

integral returning 1 means measuring on same signal, zeros mean signals not correlated.

Uncorrelated signals show no intermodulation.

4.- while time alignment kept

note that I have carefully chosen [t1 t2] integration interval to include pi. it is laborious but possible to translate chosen t1 t2 to define f1 f2, so that given f1 t1, directly choose f2(f1) t2(t1) and then repeat the above steps, obtaining same result: there's a series of frequencies above f0 that show very low intermodulation with f0, when integrating within certain time interval.

To show the importance of time alignment for the integral to measure signals orthogonality.

There's a simpler example with the available orthogonality measurement in Mathworks website using Jacobi polynomials https://uk.mathworks.com/help/symbolic/sym.jacobip.html?s_tid=srchtitle_orthogonal%2520jacobi_1

syms z
a = 3.5;
b = 7.2;
P3 = jacobiP(3, a, b, z);
P5 = jacobiP(5, a, b, z);
w = (1-z)^a*(1+z)^b;
int(P3*P5*w, z, -1, 1)

ans = 0

this integral is null therefore Jacobi polynomials P3 and P5 are orthogonal within [-1 1]

but the same polynomials are not orthogonal when using for instance [-.5 1.5]

abs(eval(int(P3*P5*w, z, -.5, 1.5)))

ans = 9.374617676947529e+05

Same functions, different integration interval, then different orthogonality results.

5.- it works even standing light time jitter

As long as phase noise kept below certain threshold now here I simulate time noise slightly shaking t1 and t2 but making sure that time jitter on start stop of the interval cycle is kept below 1us

L1=[];  % logging integral result

for k=1:1:numel(n1)

    f=[f1 f2(k)];

    dt1=randi([1000 9999],1,2)/1.7e7; % time jitter below 1us
    
    t1=-nw*pi/(2*pi*f(1))+dt1(1); % time interval start
    t2=nw*pi/(2*pi*f(1))+dt1(2); % time interval stop

    fs=160*f(2);Ts=1/fs;dt=Ts; % sampling

    t=[t1:dt:t2]; % time reference

    a=2*pi*f'*t; % constant phases
    x1=A(1)*sin(a(1,:));
    x2=A(2)*sin(a(2,:));
    xp=x1.*x2;
    L1=[L1 abs(trapz(t,xp))];

end

L1=L1/L1(1);
L2=log10(L1);

figure(5)
plot(n1,L2)
axis([1 6 -4 .1])
xlabel('n1')
grid on
title(['signal orthogonality with time jitter below 1\mus'])

As long as phase noise kept below certain threshold RF carriers can be placed in frequency close one another.

Limitations:

System clocks: both in transmitters/base stations and receivers have to be stable enough not to introduce time jitter above threshold. DVB-T uses GPS reference acquired with GPS antenna in transmitting sites and relayed to receivers in the transport stream.

Mobile communications do alike, in fact 3G started including OFDM in standards when DVB-T was already operative world-wide.

Carriers modulation: QAM QPSK and the lot allow many constellation points per symbol. More data per symbol means higher data rate, but at the expense of carriers widening. Wider carriers mean that despite achieving orthogonality on f0, the surrounding of f0 now contains significant energy, therefore intermodulation takes place. Choose a modulation that at least does not clutter the 1st zero shown in above graph.

Transmitter-Receiver distance: In-doors in-same-room are relatively easy to control RF channels in comparison to reaching hundreds of receivers scattered within a 3km radius, particularly in urban environments. Shadow points happen, and the longer an RF signal has to travel, the higher the jitter is introduced, despite system clocks fine and correct modulations.

To avoid fading-related problems, DVB-T has the Guard Interval, for each sent symbol:

There's a trailing time in each symbol that is ignored by receivers.

It can be 1/4 of the symbol time cycle ('noisy' channels) or just 1/32 ('quiet' channels). Data from the useful section is repeated into the guard interval, but receivers do not use it.