I'm looking at some code that matches audio templates in a longer audio file. The calculation correlates the power spectra of the template and audio file, maximizing over the possible alignments. This seems sub-optimal to me, because by going over to the power spectrum we're throwing away phase information. Yet when I try doing a full correlation I get worse results. I'm not 100% sure yet my implementation is correct; I could have made a silly mistake.

Is there any reason it could be better correlating the power spectra than doing a full correlation? E.g. is it more noise resistant? (I don't see why it would be. As it happens, I do have something resembling white noise in my test data.)

The only obvious thing I can think is that the power spectral correlation is probably better if the alignment error is >= 1/highest frequencies in signal because then the template and signal will be out of phase. I don't think this should be the case for me: I'm optimizing to within more temporal precision than that.

Other ideas?

EDIT: in view of Peter K's comment, I should clarify that I'm using the short time Fourier transform, summing over windows of size around 0.01s. That's how the alignment dependence enters.


If you're cross correlating short time power spectra and disregard phase, then you're not really disregarding phase.

Phase of the global Fourier transform encodes the temporal structure, including the position of the transients, evolution of tones or the local incoherence of noise.

By capturing the time dependence in the moving window of the time-frequency power spectrum, the really important aspects of phase are represented in the temporal evolution of the frequency power density.

So what you are discarding is merely "local phase", which contains a lot of information that is however modified by even the most subtle processes like sound propagation, speaker reproduction, microphone recording, etc. These modifications don't affect the qualitative content of the sound a lot, and a robust sound recognition algorithm should be mostly insensitive to them.

That means discarding the local phase will make your recognition or correlation algorithm more robust and avoid misclassifications due to small inaudible phase errors, while at the same time preserving the total temporal structure of the signal.

  • $\begingroup$ Thank you; that's very helpful. Further question then: is this the standard way of being robust against phase errors? I haven't seen it written anywhere: what I've read so far has merely suggested calculating the usual cross-correlation. My hunch is that there is some extra robust information in this local phase that isn't in the power spectrum, though of course it depends on how the phase errors can occur. (Let's say robust against phase errors that are human inaudible.) $\endgroup$ – correlator Sep 2 '15 at 18:02

I'm confused by what you're doing so I thought I'd write it out.

What you're saying is that you have a power spectral template $|X_{T}(\omega)|^2$ that you use to find a signal $x(t)$ that is at some unknown delay, $\tau$, in a time series corrupted by noise $n(t)$:

$$ y(t) = x(t-{\tau}) + n(t) $$

The technique you're using to find $\tau$ is:

$$ \tau_{\tt max} = \arg\max_{\tau} \rho(\tilde{\tau}) $$

where $\rho$ is the "correlation of power spectra": $$ \rho = \int |X_T(\xi)|^2 |Y(\xi + \omega; \tilde{\tau})|^2 d\xi $$ and $Y(\omega; \tilde{\tau})$ is the spectrum (not power spectrum) of $y(t)$ shifted by $\tilde{\tau}$.

The thing is that $\rho$ is now a function of $\omega$. And the power spectrum $|Y(\xi + \omega; \tilde{\tau})|^2$ should not depend on $\tilde{\tau}$ at all.

Can you shed some light on what I'm missing about the algorithm?

  • 1
    $\begingroup$ I think he's using time-frequency power spectral density. $\endgroup$ – Jazzmaniac Sep 2 '15 at 13:01
  • $\begingroup$ @Jazzmaniac Probably, but that's not in the question as far as I can see... $\endgroup$ – Peter K. Sep 2 '15 at 13:09
  • 1
    $\begingroup$ Hi Peter K, thanks for your answer. You have a good point. I left out a detail I didn't think relevant but actually it is in view of your comment. I'm using the short time spectrum with windows of size around 0.01s. Given an alignment, you work out the integral you mentioned and sum over windows. If you were doing this just for the spectrum (rather than the power spectrum) the result should be more or less the same as the (time domain) cross correlation, by the unitarity of the Fourier transform. With the power spectrum you lose phase info, but it depends on the alignment due to the windowing. $\endgroup$ – correlator Sep 2 '15 at 13:17
  • $\begingroup$ OK, that makes more sense. How long is the template? 0.01s? 0.1s? 1s? BTW: Welcome to DSP.SE! Nice question. :-) $\endgroup$ – Peter K. Sep 2 '15 at 13:20
  • $\begingroup$ Thank you; I'm glad to be here. The templates are of the order of 1s. $\endgroup$ – correlator Sep 2 '15 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.