0
$\begingroup$

I have a sensor which I am using to track a strong but noisy varying signal. When the signal is present there is a lot of energy in the FFT bins, when there is no signal the is much less energy - but still not zero.

I am wondering what is the correct way to use the no-signal FFT data (at startup I know there is no signal) to determine whether there is actually a signal at a later time.

Currently I am creating a noise reference by taking an FFT of the background noise and averaging it across all the bins. I am then using that reference via simple comparison to determine whether a bin contains a signal or not.

This works ok, but not perfectly and I am wondering whether calculating the Power Spectral Density of the noise and then comparing that bin-by-bin with the PSD of the signal would give better results. I would plan to scale the PSD as per https://holometer.fnal.gov/GH_FFT.pdf to eliminate the effects of the Hanning window that I am using.

I can't use ensemble averages as time resolution is important, although I could generate an ensemble average of the noise at startup if that would help.

Update: I found this paper: https://pdfs.semanticscholar.org/e59e/303e8f8fca708fe557bc36babb254ffa07f8.pdf which seems almost exactly what I want but too complex. It seems to suggest finding a noise floor by taking the minimum PSD from the spectra and then doing a direct comparison with the PSD of the signal - but clearly its got to be more compicated than that. Could I use this to calculate SNR of the peak bin and configure the minimum acceptable value?

Improve this question
$\endgroup$
5
  • $\begingroup$ The answer depends on the statistics and what you know about your signal--Does it occupy any particular spectral location or just raise the entire noise floor? Do you know anything about your signal that you can correlate to? Otherwise it is probability of false alarm vs probability of detection problem where you minimize one once setting a criteria on the other and the threshold would be N sigma of your noise floor as you suggested. If you have knowledge of your signal, then there is a lot more you could do. $\endgroup$
    – Dan Boschen
    Commented Sep 4, 2019 at 23:19
  • $\begingroup$ When there is a signal it is very strong, but somewhat noisy - it varies between about 80Hz and 350Hz but can occupy a band of 40Hz or so. When I look at the power spectrum and adjust for the window the noise is close to zero, so I am thinking of just comparing the power in the signal with the rest of the spectrum $\endgroup$
    – Andy Piper
    Commented Sep 5, 2019 at 7:45
  • $\begingroup$ So I tried that and the results are unsatisfactory when there is no signal. What I observe is that there is consistent but random level of noise in the low frequency bins, if I take the SNR of these relative to the rest (which are close to zero) I get a high SNR, even though there is no signal. I am thinking I still need to do an ensemble mean of the noise and subtract that from the SNR calculation. $\endgroup$
    – Andy Piper
    Commented Sep 5, 2019 at 9:09
  • $\begingroup$ Given what you said, and assuming that is all you know about the signal, then I suggest using an FFT with a bin width of approximately 40 Hz (the bin width is the sample rate divided by the number of samples) and then just do a threshold detection on all the bins between 80 Hz and 350 Hz, when one passes your threshold, declare signal present. As you increase your threshold, you will trade probability of false alarm with probability of detection, which you can analytically determine if you experimentally determine the statistics of the average level of each bin with and without signal present $\endgroup$
    – Dan Boschen
    Commented Sep 5, 2019 at 11:50
  • $\begingroup$ Maybe you should try some energy-based voice activity detection (VAD) methods. I don't think you have to do a FFT. However, it all depends on the spectral distrubution of the noise to be detected. $\endgroup$
    – ZR Han
    Commented Jan 29, 2021 at 1:46

1 Answer 1

Reset to default
1
$\begingroup$

It's interesting that there are no slam-dunk answers here - I think that's good, it means that there is no right answer and what works is going to hinge on the specifics of my implementation. Here is what I did:

  • I switched to measuring the power spectrum normalized for the Hanning window. This seems to give me a more stable center frequency and reduces the noise when there is no signal

  • I measured the noise on a per-bin basis and recorded the power spectrum and then averaged the output over a series of periodigrams. Since this reduces the gaussian noise I then multiplied by sqrt(N) where N was the number of periodigrams that I collected. I realise that this will amplify the non-random noise, but it also gives me a reasonable noise floor for the gaussian noise

  • I compared the noise power with the signal noise, converted to dB and allowed this to be configured by the user. 25dB works well in my case. I special-cased the situation where the noise was < 1 and made this the floor

This seems to work pretty well.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.