What Does It Mean Exactly When Two Parts of a Signal Are Correlated?

Question

I stumble quite often upon the notion that two or more parts of a signal are correlated to describe semi-formally that they belong together. For example in image processing, two pixels on an edge feature tend to be correlated whereas two adjacent parts of a 3D structure that represents water droplets in a particle simulation are less correlated. My question is what is the exact idea behind this notion.

Answer 1

Yeah, it can mess you up pretty badly if you don't get the fundamentals right off the get-go. This is how I interpret correlation, and it has worked for me for what I do for a living.

Let's start off with a relatively simple example. Take a look at the following figure (pulled from dspguide... this is actually a great online book for knowing the basics of DSP).

We have an antenna that transmits a short burst of radio wave energy in some direction. If the propagating wave strikes an object.... like a helicopter in this figure, a small fraction of the energy is reflected back toward a radio receiver. This receiver is close to the transmitting antenna.

This short burst of radio energy, for the sake of this example, is a small triangular shape. When the signal is reflected off of the helicopter, and then echoed back to the receiver, this signal will consist of two parts:

1. A shifted and scaled version of the transmitted pulse, and
2. Random noise, resulting from interfering radio waves, thermal noise in the electronics and other factors.

Loosely speaking, we can actually figure out how far the object is by using this concept. Since radio signals travel roughly at the speed of light, the shift between the transmitted and received pulse is a rough measure of the distance to the object being detected.

As such, this is our general problem:

Given a signal of some known shape, what is the best way to determine where (or if) the signal occurs in another signal?

The best way to answer this is correlation.

There are two different paradigms for computing correlation. The first one is called auto-correlation, where you are comparing a signal with shifted time offsets of itself. This paradigm that we are describing (also seen in the figure) is defined as cross-correlation, where we are comparing with another signal, notably the received signal. We essentially are comparing the received signal with shifted versions of the original transmitted signal. Basically, we take a look at what we have received and what was transmitted. We take what was received, and time shift the original transmitted signal over by different time values. We then do a comparison with each of these signals and the received result. Whichever gives us the highest value will denote how far away the helicopter is.

The amplitude of each sample in the cross-correlation signal is a measure of how much the received signal resembles the target signal, at that location. This means that a peak will occur in the cross-correlation signal for every target signal that is present in the received signal. In other words, the value of the cross-correlation is maximized when the target signal is aligned with the same features in the received signal.

If there is noise on the received signal, there will also be noise on the cross-correlation signal. It is an unavoidable fact that random noise looks a certain amount like any target signal you can choose. The noise on the cross-correlation signal is simply measuring this similarity. Except for this noise, the peak generated in the cross-correlation signal is symmetrical between its left and right. This is true even if the target signal isn't symmetrical.

A good thing to remember is that the cross-correlation is trying to detect the target signal, not recreate it. There is no reason to expect that the peak will even look like the target signal. Correlation is the optimal technique for detecting a known waveform in random noise. To be perfectly correct, it is only optimal for random white noise. Using correlation to detect a known waveform is frequently called matched filtering.

---

tl;dr - Correlation is a measure of how much one signal resembles another. The signal can be images, features, edges, etc. It is simply a measure of resemblance between one signal and another.

Answer 2

Usually this refers to the autocorrelation coefficient.

Consider any 1D signal with periodicity $\pi$.

Now let's look at the autocorrelation integral:

$$R(\tau)=\int_{-\infty}^{\infty}\! f(t)f(t-\tau)\,\mathrm{d}t$$

For varying $\tau$, the autocorrelation will have a maximum for $\tau$ equalling $\pi$ and its multiples. Thus autocorrelation can be used to study the periodicity of a signal.

This is often sort of colloquially used to indicate that certain parts of a signal are very similar or even identical.

The analogue for two different signals would be the cross correlation. It can be used to study the similarity of two separate signals.

$$(f \star g)(\tau) = \int_{-\infty}^{\infty}\! f(t)g(t-\tau)\,\mathrm{d}t$$

In case of the cross correlation $\tau$ has no significance over periodicity of the single signals but if for a given $\tau$ the correlation is high, $\tau$ indicates the phase shift between the signals.

Answer 3

Correlation between 2 signals means you can say something about one of them by observing the other.

If you mean the standard correlation, $ E \left[ x y \right] $, it means you knowledge second moment statistics.
Which implies one can, using only linear functions, estimate one from the other to some level.