2
$\begingroup$

Definition of cross correlation is

$$ \int_{\mathbb{R}}s_1^*(\tau)s_2(t+\tau)d\tau $$

I wonder using such definition, when two signals are not correlated? My guess is the following... defining $$ s_{2,\tau}(t) = s_2(t+\tau) = \left(T_{-\tau}s_2\right)(t) $$

We have

$$ \langle s_1,T_{-\tau}s_2 \rangle = 0 $$

for all $\tau$, namely $s_1$ is orthogonal to all the signals $s_{2,\tau}$. Is this guess correct?

This definition is taken from Wikipedia.

As assumption you can assume $s_1,s_2 \in \mathcal{L}^2(\mathbb{R})$, the definition should be then well posed by the Cauchy-Schwarz inequality.

$\endgroup$
10
  • 3
    $\begingroup$ Strictly speaking, the definition of cross-correlation is $\rho_{s_1s_2}(\tau) = E[s_1^*(t) s_2(t+\tau)]$ where $E$ is the expectation operator and they are uncorrelated if $\rho_{s_1s_2}(\tau) = 0$. Your "definition" only applies if you make various assumptions about $s_1$ and $s_2$. $\endgroup$
    – Peter K.
    Feb 21, 2018 at 13:13
  • $\begingroup$ You're talking about stochastic processes, that's different. I'm assuming of course deterministic signals. $\endgroup$ Feb 21, 2018 at 13:30
  • $\begingroup$ Then you should state that explicitly in your question. There are other problems with your "definition" : You need to impose some constraints on $s_1$ and $s_2$. For example, if $s_1 = s_2 = 1$ then the integral blows up. Closing this until you tidy the question up. $\endgroup$
    – Peter K.
    Feb 21, 2018 at 13:55
  • $\begingroup$ It's not "my definition". It's wikipedia definition... Check if the question is stated better please. $\endgroup$ Feb 21, 2018 at 13:58
  • 1
    $\begingroup$ @user8469759 One thing you may be missing is that the correlation is a function of $t$. So, iff the correlation integral is zero for a given $t$, then the signals are uncorrelated for that $t$. $\endgroup$
    – MBaz
    Feb 21, 2018 at 16:52

4 Answers 4

6
$\begingroup$

Cross correlation is a measure of similarity between two signals, where one signal is allowed to be time-shifted. In this sense, the correlation is not a single number, but a function of the time shift. We say, "these two signal have a certain correlation $R(\Delta)$ for a time shift $\Delta$".

Intuitively, two signals whose signs tend to have a consistent relationship (both positive or both negative, or one positive and the other negative) for a given time shift $t$ are similar, and will have large correlation (positive or negative). Signals that are uncorrelated are just as likely to have opposing signs, and then the integral will be small.

As an example, consider two signals that are rectangular pulses of duration $T=1$, but one starting at $t=0$ and the other at $t=10$. Their product is zero (and thus their correlation for time shift 0 is also 0). However, intuitively we could say that they are similar, except for the delay. Thus, the correlation allows us to quantify that intuition: these two pulses are completely uncorrelated for some delays; however, when if the first is time-delayed by 10 seconds, then they become highly correlated.

Note that $R(\Delta)=0$ is a sufficient and necessary condition for the two signals to be "uncorrelated" (meaning they have zero correlation) with time shift $\Delta$.

$\endgroup$
4
  • $\begingroup$ All this is fine, but the OP is using uncorrelated to mean that $R(\tau)$ is $0$ for all $\tau$ whereas you are meaning $R(\tau)=0$ for one specific value of $\tau$. $\endgroup$ Feb 5, 2022 at 20:37
  • $\begingroup$ @DilipSarwate Reading this four years later, I agree with you. I probably based my answer more on the OP's own comments than the question as was it was asked. $\endgroup$
    – MBaz
    Feb 5, 2022 at 21:42
  • $\begingroup$ @DanBoschen Right, as long as they are consistently opposite. The correlation metric is small when the sign combinations cancel each other out (e.g. when the signals are ++ and then +-). Does that make sense? $\endgroup$
    – MBaz
    Feb 6, 2022 at 15:41
  • 1
    $\begingroup$ @DanBoschen Fixed :) Let me know if it still needs improving. $\endgroup$
    – MBaz
    Feb 6, 2022 at 16:19
4
$\begingroup$

While it may seem overly formal, this is the best way I know of to view this issue of correlation regarding deterministic or random signals. I'm gonna change the semantics a little.

If $x(t)$ and $y(t)$ are known signals, the cross-correlation between them is

$$\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \lim_{T \to \infty} \frac{1}{T} \int\limits_{-\frac{T}{2}}^{+\frac{T}{2}} x(t) \overline{y(t+\tau)} \, \mathrm{d}t \end{align}$$

$\overline{y(t)}$ is the complex conjugate of $y(t)$. Don't worry about it if $x(t)$ and $y(t)$ are real. So this inner product notation, $\langle a,b \rangle$ is somehow multiplying $a$ and $b$ together and getting an average or mean of that result. For deterministic signals, the mean is done by that integral above (or a summation for discrete-time signals).

For random signals that mean is done probabilisticly

$$\begin{align} R_{xy}(\tau) &\triangleq \big\langle x(t),y(t+\tau) \big\rangle \\ &= \mathbb{E}\big\{ x(t) \cdot \overline{y(t+\tau)}\big\} \\ &= \int\limits_{-\infty}^{+\infty} \int\limits_{-\infty}^{+\infty} (\alpha \cdot \overline{\beta}) \ p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta \end{align}$$

where $t$ is an arbitrary time picked at random and $p_{xy}(\alpha,\beta, \tau)$ is the joint p.d.f. of random variables $x(t)$ and $y(t+\tau)$ at some arbitrary time $t$. If the differentials $\mathrm{d}\alpha$ and $\mathrm{d}\beta$ are small, then

$$ p_{xy}(\alpha,\beta,\tau) \, \mathrm{d}\alpha \, \mathrm{d}\beta = \mathrm{Probability} \Big\{ \alpha \le x(t) < \alpha + \mathrm{d}\alpha \ \mathrm{and} \ \beta \le y(t+\tau) < \beta + \mathrm{d}\beta \Big\} $$

Now, if we make an additional salient assumption and call both processes, $x(t)$ and $y(t)$ ergodic, then we are assuming that the time average for $R_{xy}(\tau)$ at the top is equal to the probabilistic average for $R_{xy}(\tau)$ at the bottom.

If $R_{xy}(\tau) = 0$ for some $\tau$, then $x(t)$ and $y(t)$ are uncorrelated (for that value of $\tau$). But that doesn't mean that they are independent random variables.

These two signals are uncorrelated (for $\tau=0$) but not independent:

$$\begin{align} x(t) &= \cos(t) \\ y(t) &= \sin(t) \\ \end{align}$$

where $t$ can be a lotta different things, including an uniform p.d.f. random variable with p.d.f. having width $2 \pi$.

$\endgroup$
2
  • $\begingroup$ Everything you say is correct, but none of it is an answer to the question the OP has asked. $\endgroup$ Feb 5, 2022 at 20:43
  • $\begingroup$ naw, not everything is correct. kinda hard to do inequalities like $$ \beta \le y(t+\tau) < \beta + \mathrm{d}\beta $$ when any of the quantities are complex. $\endgroup$ Feb 6, 2022 at 4:17
4
$\begingroup$

The cross-correlation function $R_{x,y}(\tau)$ of two deterministic finite-energy signals $x(t)$ and $y(t)$ can be defined as $$R_{x,y}(\tau) = \int_{-\infty}^\infty x^*(t)y(t+\tau) \,\mathrm dt,~-\infty < \tau < \infty, \tag{1}$$ though some folks might prefer to change the integrand to $x(t)y^*(t+\tau)$ or to negate $\tau$ (or both) to get their favorite definition. Now, an extension of Plancherel's Theorem tells us that $$\int_{-\infty}^\infty g^*(t)h(t) \,\mathrm dt = \int_{-\infty}^\infty G^*(f)H(f) \,\mathrm df \tag{2}$$ where $G(f)$ and $H(f)$ are the Fourier transforms of $g(t)$ and $h(t)$ respectively. Setting $g(t) = x(t)$ and $h(t) = y(t+\tau)$ in $(2)$, we see that we can write $$R_{x,y}(\tau) = \int_{-\infty}^\infty X^*(f)Y(f)\exp(j2\pi f\tau) \,\mathrm df \tag{3}$$ which can be recognized as the Fourier integral, that is, $R_{x,y}(\tau)$ is the inverse Fourier transform of $X^*(f)Y(f)$, but most people knew that already, right?

The OP wants $R_{x,y}(\tau)$ to have value $0$ for all $\tau, -\infty < \tau < \infty$ and it is easy to see that one way of ensuring this it to require $X^*(f)Y(f)=0 ~\forall f$. This is satisfied if $X(f)$ and $Y(f)$ have disjoint supports: for each real number $f$, at most one of $X(f)$ and $Y(f)$ is nonzero. Thus, bandlimited signals with non overlapping spectra are uncorrelated in the sense used by the OP.

$\endgroup$
-1
$\begingroup$

$$ E\{ xy\}= E\{ x \} E\{y\} $$ if $$ p(x,y)=p(x) p(y) $$

The use of the word “uncorrelated” needs to be taken in context. The Wikipedia article isn’t clear that in most contexts, the probabilistic (statistical) definition, ( usually unnormalilized) is probably more common in Signal Processing than the flipped convolution definition that is claimed to be the Signal Processing definition.

The use of the word “uncorrelated” for deterministic signals has no meaning. We don’t say signals are unconvulated. Convolution and Correlation are mathematical operations, not properties.

For random signals, uncorrelated usually means independent because $E\{ x \}$ and/or $E\{y \}$ are often zero.

You can also have $E\{ xy \}=0$ which means that the variables are orthogonal and are not independent.

Orthogonality and Independent zero mean are very different concepts which is why uncorrelated is a term, while commonly used, is often confused, particularly because there are usually better terms that can be used instead, and I haven’t even touched on the terms “covariance” and “correlation”.

$\endgroup$
5
  • $\begingroup$ independent random variables are uncorrelated, but that is a stronger attribute. not all uncorrelated random variables are independent of each other. just need to emphasize your word "usually". i might change it to "often" or "sometimes". $\endgroup$ Feb 21, 2018 at 18:47
  • $\begingroup$ Something that is “uncorrelated” but not independent is probably better called “orthogonal” or something else. I would ague that independence is a clearer attribute than the semantic ambiguity of sticking un in front of correlation, often without saying something is zero mean, and alluding to some sort of mathematical rigor, which is most often unclear, particularly to students. $\endgroup$
    – user28715
    Feb 21, 2018 at 19:17
  • $\begingroup$ @StanleyPawlukiewicz I like your answer because you specify they're not properties but mathematical operations. What's the meaning of the output of that operation then? $\endgroup$ Feb 22, 2018 at 10:27
  • $\begingroup$ @user8469759 Well, we do things like filtering for a reason. If that reason has meaning, kind-of depends $\endgroup$
    – user28715
    Feb 22, 2018 at 16:26
  • $\begingroup$ I would argue that "uncorrelated" for deterministic signals has a lot of meaning. We use this to describe code sets for CDMA where the codes are "pseudo-random" but certainly deterministic and uncorrelated. Note that the receiver uses a "correlator" with the deterministic code sequence. Also in many applications the fact that $E{x}$ is zero or not has little to do with being uncorrelated, we often exclude the DC term (such as in computations of standard deviation and variance). This would need to be stated explicitly but alone is not a good condition to establish independence vs uncorrelation $\endgroup$ Feb 26, 2022 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.