1
$\begingroup$

I collect samples (time series) at a frequency. The sample rate is arbitrary (I can change it to whatever I want.) I want to determine if what's at this frequency is a signal, or (normally distributed) noise.

I've tried using a normality test (from scipy); that didn't seem to work - there were no big differences in the results when what was at the frequency was a man-made signal (center frequency of an FM broadcast station, for example) vs. noise. I tried using auto-correlation using Pandas - Pandas always thought it was noise.

Are my techniques off? Perhaps the techniques are good, but I need to sample different (I've been sampling about every two seconds, for, say, 100 samples)? Something else?

Improve this question
$\endgroup$
1
$\begingroup$

The usual scenario is where you want to find if the received signal is noise: $$r(t)=n(t),$$ or if it is a signal plus noise: $$r(t)=s(t)+n(t).$$ It is generally assumed that you know $s(t)$. For example, you could have a radar application where you transmit a pulse and need to find if it was reflected back; or a digital communications receiver, where you want to see if the signal received was $p(t)$ (say, indicating a bit 0) or $q(t)$ (a bit 1).

The way to do this is by calculating the correlation between $r(t)$ and $s(t)$. The idea is that if the signal is present, you have \begin{align} \text{corr}(r(t),s(t)) &= \text{corr}(s(t)+n(t),s(t))\\ &=\text{corr}(s(t),s(t))+\text{corr}(s(t),n(t)) \end{align} which is large. If the signal is not present, then the correlation is small.

This scheme also works in the digital domain, assuming that you sample fast enough. In general you want to calculate the correlation of a large number of samples, so that (by the law of large numbers) the probability of being unlucky and getting "bad" noise samples is small.

In your question, you don't say if you know what the signal is. If you don't, but you know some of its properties, you may be able to look for it. For instance, if the noise is zero mean and the signal is not; or if you know that the signal's energy is concentrated in a certain band (the noise is flat). However, a specific answer is impossible without more information.

If you don't know anything about the signal, detecting it may well be impossible.

Improve this answer
$\endgroup$
  • $\begingroup$ "detecting it may well be impossible" - the thing is, I can see it with my eyes. Take the FM broadcast band for example. I see peaks at the center freqs, that taper down quickly. So I know there's something there that's not noise. I don't know the origin, modulation, or anything else; I just know there's a spike there. Why can't I do that with math? $\endgroup$ – HH- Apologize to Carole Baskin Aug 5 '17 at 4:27
  • $\begingroup$ All you need to do is quantify what you see, and then develop an algorithm to detect it. For instance, in your example, you could bandpass filter the peaks, calculate the energy of the output, and see it it's above a certain threshold. $\endgroup$ – MBaz Aug 5 '17 at 14:50
1
$\begingroup$

If you don't know your signal, you need to know your noise, and if your noise is nearly stationary, you should look at power- law detectors. The basic detector, works on the magnitude square of DFT bins. Peter Willet at UCONN is a coauthor on the majority of papers. It has its origins in passive SONAR, but seems to be used a lot in cognitive radio.

Improve this answer
$\endgroup$
  • $\begingroup$ I am assuming the noise is gaussian.... not sure how accurate that is but it's an RF noise floor. $\endgroup$ – HH- Apologize to Carole Baskin Aug 4 '17 at 16:30
  • $\begingroup$ Power-law doesn't strongly depend on strictly Gaussian noise. You need a way to estimate the noise power in your DFT bin. The basic idea is that there is more energy than would be expected. When we don't know much about a signal, we at least know that it has energy. Once you capture the signal, you need additional tests to further distinguish it $\endgroup$ – user28715 Aug 4 '17 at 16:47
  • $\begingroup$ It sounds like a basic method would be to take the DFT in all bins, calculate the average power and power standard deviation, and assuming that there is much more noise floor than signal, look for deviation from this average. One problem with this in my case is that there are big discontinuities in the noise floor (e.g. from multiple antenna inputs for different frequency bands.) $\endgroup$ – HH- Apologize to Carole Baskin Aug 4 '17 at 19:48
  • $\begingroup$ Actually it's a little different, but discontinuous noise would be a problem. A problem that also haunts any of what you mentioned that you tried. $\endgroup$ – user28715 Aug 4 '17 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.