Cross correlation, when two signals are “uncorrelated”?

Definition of cross correlation is

$$\int_{\mathbb{R}}s_1^*(\tau)s_2(t+\tau)d\tau$$

I wonder using such definition, when two signals are not correlated? My guess is the following... defining $$s_{2,\tau}(t) = s_2(t+\tau) = \left(T_{-\tau}s_2\right)(t)$$

We have

$$\langle s_1,T_{-\tau}s_2 \rangle = 0$$

for all $\tau$, namely $s_1$ is orthogonal to all the signals $s_{2,\tau}$. Is this guess correct?

This definition is taken from Wikipedia.

As assumption you can assume $s_1,s_2 \in \mathcal{L}^2(\mathbb{R})$, the definition should be then well posed by the Cauchy-Schwarz inequality.

• Strictly speaking, the definition of cross-correlation is $\rho_{s_1s_2}(\tau) = E[s_1^*(t) s_2(t+\tau)]$ where $E$ is the expectation operator and they are uncorrelated if $\rho_{s_1s_2}(\tau) = 0$. Your "definition" only applies if you make various assumptions about $s_1$ and $s_2$. – Peter K. Feb 21 '18 at 13:13
• You're talking about stochastic processes, that's different. I'm assuming of course deterministic signals. – user8469759 Feb 21 '18 at 13:30
• Then you should state that explicitly in your question. There are other problems with your "definition" : You need to impose some constraints on $s_1$ and $s_2$. For example, if $s_1 = s_2 = 1$ then the integral blows up. Closing this until you tidy the question up. – Peter K. Feb 21 '18 at 13:55
• It's not "my definition". It's wikipedia definition... Check if the question is stated better please. – user8469759 Feb 21 '18 at 13:58
• I don't understand your question. Are you asking if, given that $s_1(t)$ and $s_2(t)$ are uncorrelated (meaning that the integral is zero for all $t$), then one can conclude that $s_1(t)$ and $s_2(t-\tau)$ are also uncorrelated? – MBaz Feb 21 '18 at 15:03

Cross correlation is a measure of similarity between two signals, where one signal is allowed to be time-shifted. In this sense, the correlation is not a single number, but a function of the time shift. We say, "these two signal have a certain correlation $R(\Delta)$ for a time shift $\Delta$".

Intuitively, two signals that tend to have the same sign (both positive or both negative) for a given time shift $t$ are similar, and will have large correlation (positive or negative). Signals that are uncorrelated are just as likely to have opposing signs, and then the integral will be small.

As an example, consider two signals that are rectangular pulses of duration $T=1$, but one starting at $t=0$ and the other at $t=10$. Their product is zero (and thus their correlation for time shift 0 is also 0). However, intuitively we could say that they are similar, except for the delay. Thus, the correlation allows us to quantify that intuition: these two pulses are completely uncorrelated for some delays; however, when if the first is time-delayed by 10 seconds, then they become highly correlated.

Note that $R(\Delta)=0$ is a sufficient and necessary condition for the two signals to be "uncorrelated" (meaning they have zero correlation) with time shift $\Delta$.

$$E\{ xy\}= E\{ x \} E\{y\}$$ if $$p(x,y)=p(x) p(y)$$

The use of the word “uncorrelated” needs to be taken in context. The Wikipedia article isn’t clear that in most contexts, the probabilistic (statistical) definition, ( usually unnormalilized) is probably more common in Signal Processing than the flipped convolution definition that is claimed to be the Signal Processing definition.

The use of the word “uncorrelated” for deterministic signals has no meaning. We don’t say signals are unconvulated. Convolution and Correlation are mathematical operations, not properties.

For random signals, uncorrelated usually means independent because $E\{ x \}$ and/or $E\{y \}$ are often zero.

You can also have $E\{ xy \}=0$ which means that the variables are orthogonal and are not independent.

Orthogonality and Independent zero mean are very different concepts which is why uncorrelated is a term, while commonly used, is often confused, particularly because there are usually better terms that can be used instead, and I haven’t even touched on the terms “covariance” and “correlation”.

• independent random variables are uncorrelated, but that is a stronger attribute. not all uncorrelated random variables are independent of each other. just need to emphasize your word "usually". i might change it to "often" or "sometimes". – robert bristow-johnson Feb 21 '18 at 18:47
• Something that is “uncorrelated” but not independent is probably better called “orthogonal” or something else. I would ague that independence is a clearer attribute than the semantic ambiguity of sticking un in front of correlation, often without saying something is zero mean, and alluding to some sort of mathematical rigor, which is most often unclear, particularly to students. – user28715 Feb 21 '18 at 19:17
• @StanleyPawlukiewicz I like your answer because you specify they're not properties but mathematical operations. What's the meaning of the output of that operation then? – user8469759 Feb 22 '18 at 10:27
• @user8469759 Well, we do things like filtering for a reason. If that reason has meaning, kind-of depends – user28715 Feb 22 '18 at 16:26

While it may seem overly formal, this is the best way I know of to view this issue of correlation regarding deterministic or random signals. I'm gonna change the semantics a little.

If $x(t)$ and $y(t)$ are known signals, the cross-correlation between them is

\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \lim_{T \to \infty} \frac{1}{T} \int\limits_{-\frac{T}{2}}^{+\frac{T}{2}} x(t) \overline{y(t+\tau)} \, \mathrm{d}t \end{align}

$\overline{y(t)}$ is the complex conjugate of $y(t)$. Don't worry about it if $x(t)$ and $y(t)$ are real. So this inner product notation, $\langle a,b \rangle$ is somehow multiplying $a$ and $b$ together and getting an average or mean of that result. For deterministic signals, the mean is done by that integral above (or a summation for discrete-time signals).

For random signals that mean is done probabilisticly

\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \mathbb{E}\big\{ x(t) \cdot \overline{y(t+\tau)}\big\} \\ &= \int\limits_{-\infty}^{+\infty} \int\limits_{-\infty}^{+\infty} (\alpha \cdot \overline{\beta}) \ p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta \end{align}

where $t$ is an arbitrary time picked at random and $p_{xy}(\alpha,\beta)$ is the joint p.d.f. of random variables $x(t)$ and $y(t)$ at that arbitrary time $t$. If the differentials $\mathrm{d}\alpha$ and $\mathrm{d}\beta$ are small, then

$$p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta = \mathrm{Probability} \big\{ \alpha \le x(t) < \alpha + \mathrm{d}\alpha \ \mathrm{and} \ \beta \le y(t+\tau) < \beta + \mathrm{d}\beta \big\}$$

Now, if we make an additional salient assumption and call both processes, $x(t)$ and $y(t)$ ergodic, then we are assuming that the time average for $R_{xy}(\tau)$ at the top is equal to the probabilistic average for $R_{xy}(\tau)$ at the bottom.

If $R_{xy}(\tau) = 0$ for some $\tau$, then $x(t)$ and $y(t)$ are uncorrelated (for that value of $\tau$). But that doesn't mean that they are independent random variables.

These two signals are uncorrelated (for $\tau=0$) but not independent:

\begin{align} x(t) &= \cos(t) \\ y(t) &= \sin(t) \\ \end{align}

where $t$ can be a lotta different things, including an uniform p.d.f. random variable of width $2 \pi$.