# Cross-correlation or cross-covariance of non-zero mean signals

Cross-correlation for uniformly sampled signals is defined as [1]

$$(f \star g)[n]\ \stackrel{\mathrm{def}}{=} \sum_{m=-\infty}^{\infty} f^*[m]\ g[m+n].$$

Cross-covariance for wide-sense stationary (WSS) signals is defined identically [2]. This seems strange to me, because the statistical definition [3] first subtracts the signal means from the signals before correlating it with each other. In other words, cross-covariance is substantially different from cross-correlation if the mean of the signals is substantially different from zero (is the Wikipedia article wrong? Or does it somehow assume zero mean signals?).

So, cross-covariance equals cross-correlation if the signal means are zero, else it is unequal.

Questions:

1. What are reasons to choose for cross-correlation or cross-covariance when comparing signals with non-zero mean?
2. Does cross-correlation have any significance in this case, because $f\star g$ has mean $\mu_f\mu_g$?
3. How would one find out the cross-correlation of signals that are not WSS, e.g. the mean is a linear trend? What would happen if we'd ignore this linear trend and simply subtract the signal means, then calculate cross-correlation? What if we subtracted the linear trend instead of the mean? What are the consequences for such mild violations of WSS-ity?
• Your question is great! The widespread confusion over the sample covariance vs the statistical covariance is really, really bad. See, for example, these recent posts: post 1 and post 2. – Peter K. Oct 13 '16 at 16:53
• Thanks for the compliment! I was kinda hoping for this comment. I thought this too, but I wasn't so sure the confusion was as widespread as you claim. I hope someone can answer my questions while shedding some light on the matter of the similarities and differences. – Erik Oct 13 '16 at 17:09
• Erik, I'm just trying to write as complete an answer as I can. It'll be a while as my day job is getting in the way. :-) – Peter K. Oct 13 '16 at 17:43

What are reasons to choose for cross-correlation or cross-covariance when comparing signals with non-zero mean?

Well, part of the issue is that cross-correlation as defined in your equation:

$$(f \star g)[n]\ \stackrel{\mathrm{def}}{=} \sum_{m=-\infty}^{\infty} f^*[m]\ g[m+n].$$

will not exist (or be infinite) if $f$ and $g$ have non-zero mean. So, in order to do the calculation, you really should subtract the mean beforehand.

If you're dealing with finite length signals, it's even worse, because you end up with an underlying triangular offset as per the image below.

This shows the sample cross-correlation between two gaussian processes with non-zero mean. The black line is the sample cross-correlation calculated without subtracting the mean. The red line is the sample cross-correlation calculated by first subtracting the mean from both signals.

Does cross-correlation have any significance in this case, because $f\star g$ has mean $\mu_f\mu_g$?

It doesn't generally, unless you're very interested in the mean value.

And, as I said, as per your definition, any value of $\mu_f$ and $\mu_g$ both different from zero will result in an infinite sample cross-correlation.

How would one find out the cross-correlation of signals that are not WSS, e.g. the mean is a linear trend? What would happen if we'd ignore this linear trend and simply subtract the signal means, then calculate cross-correlation? What if we subtracted the linear trend instead of the mean? What are the consequences for such mild violations of WSS-ity?

When looking at linear trends, it's usual to detrend before looking at the statistics of the variation around the trend. Sometimes doing this allows the variations to be treated as WSS.

If you keep the trend, you'll end up with a sample cross-correlation that looks like a parabola (the integration of a linear trend). In this case, simply removing the mean doesn't help. See similar plot to first below, this time with two linear trend noisy signals.

Please see this exposition on the Mathworks site for a Matlab version of the above / below.

R Code Only Below

#34778
#install.packages('SynchWave')
require(SynchWave)

with_mean_ccf <- function(f,g)
{
length <- length(f) + length(g) - 1

ff <- c(f, rep(0,length - length(f) + 1))
gg <- c(f, rep(0,length - length(g) + 1))

FF <- fft(ff)
GG <- fft(gg)

ccf <- fft(FF * Conj(GG), inverse = TRUE)

return(fftshift(Re(ccf)))
}

T <- 1000
mu_f <- 1
mu_g <- 2

f <- rnorm(T,1) + mu_f
g <- rnorm(T,1) + mu_g

with_mean <- with_mean_ccf(f,g)
no_mean <- with_mean_ccf(f-mean(f), g-mean(g))
plot(with_mean)
points(no_mean, col='red')
title('Sample crosscorrelation with (red) and without (black) mean subtraction')

a <- rnorm(T,1) + seq(1,T)/3
b <- rnorm(T,1) + seq(1,T)/2

with_mean_and_trend <- with_mean_ccf(a,b)
no_mean_and_trend <- with_mean_ccf(a-mean(a),b-mean(b))

plot(with_mean_and_trend, ylim=c(min(with_mean_and_trend,no_mean_and_trend), max(with_mean_and_trend,no_mean_and_trend)) )
points(no_mean_and_trend, col="red")
title('Sample cross-correlation of two linear trend signals')

• I've accepted your answer and I've added a link to MATLAB demonstration code that performs similar calculations as your R code. The amplitude of your cross-correlations if much higher. Does this come from the way you calculate it, using fft and the inverse? Maybe you should normalise for the number of samples. Also, the zero lag cross-correlation in your figures stands out: is it double of what you'd expect? This could also be an artefact from Fourier transformation. – Erik Oct 14 '16 at 9:08
• @Erik Yes the amplitudes are because R's FFT doesn't do the same normalizaiton that Matlab's does. Normalization is a whole other argument. :-) I'll leave it as an exercise for the reader. Again, yes, the zero term is probably because of the way R normalizes (or doesn't, really) its FFTs. – Peter K. Oct 14 '16 at 11:31
• My edit was rejected, unfortunately. It was basically an addition of some MATLAB code that does the same as your R code. Find the code with results here. I think it may be practical for people using MATLAB that want to see how to perform cross-correlation analyses in some general cases. – Erik Oct 15 '16 at 14:46
• @Erik Yes, your edit was rejected before I got to it; I've added a line with that link to the answer now. – Peter K. Oct 15 '16 at 15:43