How to detect signal in spectrum without empirical data

Question

Let's say, I have got a spectrum of a signal with noise using fft:

For a human it's pretty obvious that there is a signal with a frequency of about 51Hz. But what is the mathematical apparatus behind such conclusion and is there a tool for this in python?

Easiest solution in this example would be to just take max(y) of spectrum to get the frequency of signal but it would work only in a situation where spectrum actually has desired signal.

Second way is to take coefficient of variation:

$$c_V = { \sigma \over \mu}.$$

numpy.std(abs(y0))/numpy.mean(abs(y0))

Which is definitely more preferrable, however this coefficient is rather empirical and would need some kind of threshold value.

Is there some function or formula that would make it clear if there is signal in spectrum without the need to use empirical values like cut-off frequencies or threshold for spectral density as the noise or the signal can be of any of any magnitude and frequency?

Answer 1

The general detection problem (not just some signal based on a spectrograph) demands that you start with some a-priori knowledge*, and make a detector that looks at the available data and makes a yes/no determination.

Even in machine learning, where we do not bother figuring out the general characteristics of the thing we want to detect, we still start with a-priori knowledge in the form of a giant set of tagged example data so that the machine can determine if the thing we wish to detect is there or not.

Is there some function or formula that would make it clear if there is signal in spectrum without the need to use empirical values...

Only if you know ahead of time what signal you're trying to detect, it's characteristics, and the characteristics of any interfering signals or noise. Then, if you know enough, you can construct a detector -- possibly even one that's optimal in some sense or another -- to detect your signal.

Note that even your thesis statement is, in a larger sense, impossible. For example, given a spread spectrum signal in a noisy environment you cannot tell if the signal is present or not by looking at the amplitude of a spectrogram. That's because spread spectrum means that the signal -- while present in a perfectly sensible way -- appears as a slightly raised noise floor in the spectrogram, and you can't distinguish that from, well, noise.

* a-priori knowledge == stuff you know ahead of time.

Answer 2

It is not a trivial question whether to determine which are the fundamental frequencies of a signal computationally, without visual inspection. I know from some methods that integrate the signal with a moving window to reduce the resolution and apply a thresholding technique to get all the frequencies above a threshold.

This is a problem I have faced in the past and my solution was to:

1. locate the absolute maxima of the FFT and the highest local maxima (second after absolute maxima). Keep those values.
2. Depending on the difference (that is determined by your PSD and SNR of your signal), determine the threshold to find in the spectrum. If you use the absolute maxima with respect to the baseline of your signal (mean of all possible frequencies) you might find it difficult to locate all fundamental frequencies in the spectrum.
3. Apply a peak finding algorithm (AMPD, recursive search, scipy has pretty good functionalities) with the threshold that you have chosen from the previous step.
4. Consider all the resultant peaks as your fundamental frequencies. You can establish a confidence level in your selection.

As you can imagine, there are many ways of doing this task and I am sure that someone else might have a better solution, but this is what I've done in my works.