I have the fft of some signal, and want a rough estimate of the noise level in order to choose an appropriate threshold for our peak detection algorithm. In general, the fft contains mostly noise with a handful of peaks (which are usually pretty high compared to said noise). For reference, I've attached a screenshot of a pretty typical fft:
Now to my proposed algorithm for noise level estimation, which is based on the following assumptions:
- What i want is the mean and standard deviation of the fft
- If the fft contained only (gaussian) noise, the median would be very close to the mean
- Peaks shouldn't affect the median too much
- The mean and standard deviation are independent of the order of the samples
- Therefore i can sort the fft and still get the same result
So the resulting algorithm looks like this:
- Sort the fft (ascending)
- Calculate the mean and standard deviation of the sorted fft, but stop as soon as the mean exceeds the median
- Use the calculated mean and standard deviation to choose the peak detection threshold
I've tested this algorithm on some typical signals and the results seem to be pretty decent (keeping in mind that i don't need the exact noise level, but just something robust to choose an appropriate threshold). Sorting is a little expensive of course, but i don't expect the noise level to change too much, so i don't need to run it for every fft.
That being said, the algorithm itself feels kind of "wrong", because i don't ask the actual question "What would the mean and standard deviation be without the peaks?" but rather "What would the standard deviation be if the mean was X?".
So what are your thoughts on this? Are there better algorithms for this kind of problem? (i bet there are)