Given an experimental FFT plot, how to find the best match from a set of generated FFT plots?

Question

What I'm trying to do is, given some experimental data and it's (magnitude normalized to 1) FFT, match the FFT with one of say 1000 other (magnitude normalized to 1) FFT plots. Please note that the experimental/generated plots all have the same bin width/count.

Initially what I'm doing is taking the normalized cross correlation between the experimental FFT plot and each of the generated FFT plots, and then saving only the zero lag value. Then the highest of these values is found at the end which tells which plot correlated best..

This actually works pretty well, but it doesn't seem like something that is done often.. so

1. Does this make sense? Is it something that's ever done in the DSP community? Are there better methods of doing this which I'm oblivious to? (Note I have tried an MSE metric with limited success).
2. If what I'm doing is okay, are there any improvements I can make to it. I have two small problems. The first is computational time, I can compute the zero lag correlation by hand in Matlab as sum(x.*y)/scalefactor (scalefactor = sqrt(sum(x.^2).*sum(y.^2)) ) or I can use xcorr(x,y,0,'coeff') to grab the zero lag value. I'm not sure if these are optimized as well as they could be... The second issue is that if any of the set of generated FFT's are large, say a constant 1 over f=0 to 0.5, the cross correlation metric will be large despite the experimental FFT being much "different". This actually makes me think cross correlation is not the right metric to use, so, any thoughts?

UPDATE: So thinking about this some more, I determined what I was doing was equivalent to just taking the two sequences and performing a circular convolution on them and saving the maximum value. This is the same as computing the FFT of the sequences and then cross correlating them and saving the zero lag value.

With this method I end up doing something like ifft(fft(x).*conj(fft(y))) to do the circular correlation quickly, although I'm still trying to determine if this is more efficient than what i was doing before...

Answer 1

Regarding the problem with high correlation values, this happens when both signals are not zero-mean.

Assume two signals $x[k]$ and $y[k]$ of length $N$ each. Each signal can be decomposed into the sum of a constant and a mean-free signal as follows:

$$x[k] = \overline{x} + x_0[k]$$

with $\overline{x}$ being the mean

$$\overline{x} = \frac{1}{N} \sum_{k=0}^{N-1}x[k]$$

and $x_0[k] = x[k] - \overline{x}$. Do the same to get

$$y[k] = \overline{y} + y_0[k]$$

Correlating both sequences at zero lag yields (ignoring scaling here, but this does not change the issue)

$$c = \sum_{k=0}^{N-1}x[k] \cdot y[k] = \sum_{k=0}^{N-1}\overline{x} \cdot \overline{y} + \overline{x} \cdot y_0[k] + \overline{y} \cdot x_0[k] + x_0[k] \cdot y_0[k]$$

Since $x_0[k]$ and $y_0[k]$ are zero-mean, the cross-terms cancel out so the final result is

$$c = N \cdot \overline{x} \cdot \overline{y} + \sum_{k=0}^{N}x_0[k] \cdot y_0[k]$$

So if both correlated signals are not mean-free, the correlation will always contain the product of both means times the amount of samples. If you don't want this and the mean is not relevant, you should subtract it prior to correlating.

Regarding the Matlab question. You can pack all reference signals next to each other in a matrix and then correlate with all at once at zero-lag. If you have $M$ reference signals of length $N$ each, you could do (assuming 'signal' as a column vector and the reference signals saved in columns of 'ref' as well)

c = sum(repmat(signal, 1, M) .* ref)

to obtain a vector row vector c containing the correlations of your experimental signal with all the reference signals ref. No need to calculate the whole correlation or convolutions which is indeed a huge overhead. If you do not want to use repmat, you can also have a look at bsxfun which can achieve the same result and is generally interesting when you want to do element-wise operations with a vector on each row or column of a matrix.

Answer 2

If the signals do not have any dominating harmonics in them and they all look similar in envelope and smooth enough, maybe you can try sampling the FFTs and performing the same(correlation metrics). It might save you some amount of computation.

And as for your question as to if you're doing something out of the ordinary, I don't think so. Cross correlation at zero lag is the best metric that can give you the closest answer.