2
$\begingroup$

I am trying to determine the main frequency of a noisy signal that varies in frequency over time. Ideally I want to detect changes in the frequency as rapidly as possible - say 50Hz update rate, but I also want to get as close to the actual signal frequency as possible. The signal I am looking for varies between 100Hz and 300Hz and I am sampling the signal at 1Khz.

Currently I am using a STFT of length N applied every N/2 samples (so hop size N/2) and using a Hanning window on the input. I am then averaging the 3 overlapping frames via Welch's method to achieve a final estimate of the signal I am looking for.

I am aware that the maximum frequency resolution I can get is 1000/N so I am using Jain's method to interpolate between bins. I am also aware that the maximum time resolution I can get is 1000/N Hz. I know that these two are in tension.

Sometimes the signal is missing. In order to detect this I am calculating the standard deviation as well as the average of the three overlapping frames and using the ratio to determine the "noisiness"

My questions are:

  • Is this a valid procedure?
  • Is using Welch in this context valid given that the input signal varies?
  • Am I averaging the right data?
  • Is the time resolution really 1000/N or is the fact that I can produce output at 1000/2N Hz better?
  • Would more overlap help? If so how?
  • Is my noise calculation valid?

I have heavily relied on https://holometer.fnal.gov/GH_FFT.pdf

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ your STFT frame hop need not be $\frac12$ of your frame width, as is often the case. you can make the frame hop 20 ms and still have a frame width of 300 ms or so. $\endgroup$
    – robert bristow-johnson
    Aug 23, 2019 at 4:00
  • $\begingroup$ Good to know. Should I still average all the overlapping frames or does this not buy me anything? $\endgroup$
    – Andy Piper
    Aug 23, 2019 at 6:24
  • 1
    $\begingroup$ i don't think it does. there is already a sense of averaging with such wide frames. i.e. transients will be smeared over multiple frames so even if the frame hop is 1/50th second, since the frame itself is so much larger and the overlap is large, the change between adjacent frames will not be great. $\endgroup$
    – robert bristow-johnson
    Aug 23, 2019 at 7:20
  • $\begingroup$ In dsp.stackexchange.com/questions/47448/… you reference COLA - is that relevant here? Should I do any averaging at all? I thought averaging reduced noise? $\endgroup$
    – Andy Piper
    Aug 23, 2019 at 8:29
  • 1
    $\begingroup$ well, i made a small modification to someone else's post there. you can do a bin-by-bin sliding average of each DFT output if you want, but that will make the time resolution even sloppier. $\endgroup$
    – robert bristow-johnson
    Aug 23, 2019 at 19:40

2 Answers 2

Reset to default
1
$\begingroup$

I think that you are splitting your signal $x[n]$ into $N$ STFT intervals which you are overlapping by 50%, then on each interval finding the Power Spectral Density (PSD) with Welch's Method.

If this is correct I think this method is valid,

but I think you might be confused by the 'overlapping' because there are two at play here. When you use Welch's method you are splitting the STFT interval into smaller intervals, from which the periodograms is calculated, and then these are averaged to give you your PSD for that STFT interval.

So there are some variables which can be changed here

  1. The length of your STFT intervals, $N$
  2. The overlap of your STFT intervals
  3. The overlap of your welch's periodograms

The smaller the length of the STFT the less smooth your 'time resolution' will be because you are averaging over a smaller length of time which will allow you to pick out sudden changes in frequencies, but less data is obviously going to have some kind of negative effect (maybe lower frequency resolution, but im not sure)

I think how much you overlap the STFT is quite related to the 'time resolution' here because you can think of it like, for the first interval you've windowed you are finding the PSD over $[x[0] - x[N]]$ but then you could quite easily take the next interval to be $[x[1] - x[N+1]]$ these two clearly overlap depending on $N$, but its better to just think of you moving your STFT window along $x[n]$ and calculating the PSD at each one, so in this sense your 'time resolution' can just be the same as your sampling rate, but its smoothed by the overlapping.

The overlapping of the periodograms smooths the frequency resolution but also reduces noise

I'm not sure what Jains method is but if you wanted to look at higher frequency resolution methods I would look at instantaneous frequencies using the Hilbert-Huang Transform, see this paper for some comparisons

Improve this answer
$\endgroup$
7
  • $\begingroup$ Yes I didn't realize that the window size (L) and STFT size (W) could be different! I read somewhere that having W greater than L by a power of two is desirable, but didn't see why. Is this the case? Also "overlapping of the periodograms smooths the frequency resolution but also reduces noise" but only if you average the periodograms, right? $\endgroup$
    – Andy Piper
    Aug 22, 2019 at 19:00
  • $\begingroup$ Yeah that's correct, but if you're using Welches method then that's what it does, what you can try to do to increase the frequency resolution is increase the amount of data in each window, either by just having fewer periodograms or zero padding the intervals within Welch's method might work but I'm not sure $\endgroup$
    – Stephen Jackson
    Aug 22, 2019 at 19:10
  • $\begingroup$ And I think that thing you read probably means you want the length of the fourier transform to be the next power of two higher than the length of the data or something to decrease the amount of time to fourier transform, have a look at matlabs nextpow2 function $\endgroup$
    – Stephen Jackson
    Aug 22, 2019 at 19:13
  • $\begingroup$ Zero-padding achieves the same as interpolation I believe, so probably will stick with the latter. Don't understand your next comment - inreasing the length of the FFT will increase the amount of time surely? $\endgroup$
    – Andy Piper
    Aug 22, 2019 at 19:28
  • $\begingroup$ Yeah that's correct I think, so if you have fewer periodograms but with more data in (or more time in each periodogram) you average less but get a higher frequency resolution $\endgroup$
    – Stephen Jackson
    Aug 22, 2019 at 19:41
0
$\begingroup$

I'm going to attempt to answer my own question based on some of the comments and my own further investigation

Is this a valid procedure?

It's not invalid, but it does not necessarily achieve the outcome I am looking for. Welch is used to improve the SNR in a DFT by averaging shorter length DFT's. The more shorter DFT's you use the better the outcome, but the converse is also true - the fewer shorter length DFT's you use the more negligible the improvement and for my case - averaging 3 periodograms, the reduction in noise is $1/\sqrt{3}$. In addition continuously averaging DFT's reduces the time resolution by increasing the latency. In my case, because time resolution is important, using Welch is essentially counter-productive.

Is using Welch in this context valid given that the input signal varies?

It is as long as the input signal is essentially stationary for the number of short DFT's in use - the more DFT's you use the less likely this is to be true.

Am I averaging the right data?

Yes, but not enough of it to make a difference

Is the time resolution really 1000/N or is the fact that I can produce 
output at 1000/2N Hz better?

I believe that the time resolution is made worse by this method

Would more overlap help? If so how?

No

Is my noise calculation valid?

Yes, but given the small number of averaged frames distinguishing noise from signal will be tricky.

So in summary, I would be better off using short-time fourier transform (STFT) and using an overlapped hopping window with interpolation to improve the update rate and increase the frequency resolution

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.