Cross correlation, when two signals are "uncorrelated"?

Question

Definition of cross correlation is

$$ \int_{\mathbb{R}}s_1^*(\tau)s_2(t+\tau)d\tau $$

I wonder using such definition, when two signals are not correlated? My guess is the following... defining $$ s_{2,\tau}(t) = s_2(t+\tau) = \left(T_{-\tau}s_2\right)(t) $$

We have

$$ \langle s_1,T_{-\tau}s_2 \rangle = 0 $$

for all $\tau$, namely $s_1$ is orthogonal to all the signals $s_{2,\tau}$. Is this guess correct?

This definition is taken from Wikipedia.

As assumption you can assume $s_1,s_2 \in \mathcal{L}^2(\mathbb{R})$, the definition should be then well posed by the Cauchy-Schwarz inequality.

Answer 1

Cross correlation is a measure of similarity between two signals, where one signal is allowed to be time-shifted. In this sense, the correlation is not a single number, but a function of the time shift. We say, "these two signal have a certain correlation $R(\Delta)$ for a time shift $\Delta$".

Intuitively, two signals whose signs tend to have a consistent relationship (both positive or both negative, or one positive and the other negative) for a given time shift $t$ are similar, and will have large correlation (positive or negative). Signals that are uncorrelated are just as likely to have opposing signs, and then the integral will be small.

As an example, consider two signals that are rectangular pulses of duration $T=1$, but one starting at $t=0$ and the other at $t=10$. Their product is zero (and thus their correlation for time shift 0 is also 0). However, intuitively we could say that they are similar, except for the delay. Thus, the correlation allows us to quantify that intuition: these two pulses are completely uncorrelated for some delays; however, when if the first is time-delayed by 10 seconds, then they become highly correlated.

Note that $R(\Delta)=0$ is a sufficient and necessary condition for the two signals to be "uncorrelated" (meaning they have zero correlation) with time shift $\Delta$.

Answer 2

While it may seem overly formal, this is the best way I know of to view this issue of correlation regarding deterministic or random signals. I'm gonna change the semantics a little.

If $x(t)$ and $y(t)$ are known signals, the cross-correlation between them is

$$\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \lim_{T \to \infty} \frac{1}{T} \int\limits_{-\frac{T}{2}}^{+\frac{T}{2}} x(t) \overline{y(t+\tau)} \, \mathrm{d}t \end{align}$$

$\overline{y(t)}$ is the complex conjugate of $y(t)$. Don't worry about it if $x(t)$ and $y(t)$ are real. So this inner product notation, $\langle a,b \rangle$ is somehow multiplying $a$ and $b$ together and getting an average or mean of that result. For deterministic signals, the mean is done by that integral above (or a summation for discrete-time signals).

For random signals that mean is done probabilisticly

$$\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \mathbb{E}\big\{ x(t) \cdot \overline{y(t+\tau)}\big\} \\ &= \int\limits_{-\infty}^{+\infty} \int\limits_{-\infty}^{+\infty} (\alpha \cdot \overline{\beta}) \ p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta \end{align}$$

where $t$ is an arbitrary time picked at random and $p_{xy}(\alpha,\beta, \tau)$ is the joint p.d.f. of random variables $x(t)$ and $y(t+\tau)$ at some arbitrary time $t$. If the differentials $\mathrm{d}\alpha$ and $\mathrm{d}\beta$ are small, then

$$ p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta = \mathrm{Probability} \Big\{ \alpha \le x(t) < \alpha + \mathrm{d}\alpha \ \mathrm{and} \ \beta \le y(t+\tau) < \beta + \mathrm{d}\beta \Big\} $$

Now, if we make an additional salient assumption and call both processes, $x(t)$ and $y(t)$ ergodic, then we are assuming that the time average for $R_{xy}(\tau)$ at the top is equal to the probabilistic average for $R_{xy}(\tau)$ at the bottom.

If $R_{xy}(\tau) = 0$ for some $\tau$, then $x(t)$ and $y(t)$ are uncorrelated (for that value of $\tau$). But that doesn't mean that they are independent random variables.

These two signals are uncorrelated (for $\tau=0$) but not independent:

$$\begin{align} x(t) &= \cos(t) \\ y(t) &= \sin(t) \\ \end{align}$$

where $t$ can be a lotta different things, including an uniform p.d.f. random variable with p.d.f. having width $2 \pi$.

Answer 3

The cross-correlation function $R_{x,y}(\tau)$ of two deterministic finite-energy signals $x(t)$ and $y(t)$ can be defined as $$R_{x,y}(\tau) = \int_{-\infty}^\infty x^*(t)y(t+\tau) \,\mathrm dt,~-\infty < \tau < \infty, \tag{1}$$ though some folks might prefer to change the integrand to $x(t)y^*(t+\tau)$ or to negate $\tau$ (or both) to get their favorite definition. Now, an extension of Plancherel's Theorem tells us that $$\int_{-\infty}^\infty g^*(t)h(t) \,\mathrm dt = \int_{-\infty}^\infty G^*(f)H(f) \,\mathrm df \tag{2}$$ where $G(f)$ and $H(f)$ are the Fourier transforms of $g(t)$ and $h(t)$ respectively. Setting $g(t) = x(t)$ and $h(t) = y(t+\tau)$ in $(2)$, we see that we can write $$R_{x,y}(\tau) = \int_{-\infty}^\infty X^*(f)Y(f)\exp(j2\pi f\tau) \,\mathrm df \tag{3}$$ which can be recognized as the Fourier integral, that is, $R_{x,y}(\tau)$ is the inverse Fourier transform of $X^*(f)Y(f)$, but most people knew that already, right?

The OP wants $R_{x,y}(\tau)$ to have value $0$ for all $\tau, -\infty < \tau < \infty$ and it is easy to see that one way of ensuring this it to require $X^*(f)Y(f)=0 ~\forall f$. This is satisfied if $X(f)$ and $Y(f)$ have disjoint supports: for each real number $f$, at most one of $X(f)$ and $Y(f)$ is nonzero. Thus, bandlimited signals with non overlapping spectra are uncorrelated in the sense used by the OP.

Answer 4

$$ E\{ xy\}= E\{ x \} E\{y\} $$ if $$ p(x,y)=p(x) p(y) $$

The use of the word “uncorrelated” needs to be taken in context. The Wikipedia article isn’t clear that in most contexts, the probabilistic (statistical) definition, ( usually unnormalilized) is probably more common in Signal Processing than the flipped convolution definition that is claimed to be the Signal Processing definition.

The use of the word “uncorrelated” for deterministic signals has no meaning. We don’t say signals are unconvulated. Convolution and Correlation are mathematical operations, not properties.

For random signals, uncorrelated usually means independent because $E\{ x \}$ and/or $E\{y \}$ are often zero.

You can also have $E\{ xy \}=0$ which means that the variables are orthogonal and are not independent.

Orthogonality and Independent zero mean are very different concepts which is why uncorrelated is a term, while commonly used, is often confused, particularly because there are usually better terms that can be used instead, and I haven’t even touched on the terms “covariance” and “correlation”.