What is the best way to decide whether a periodic signal is present in a noisy environment with a limited number of samples?

Question

Suppose I have few samples (e.g. 20) of a decaying sinusoidal function and I am heavily noise-dominated (SNR ~1, noise is Gaussian or Poissonian). Let's think of e.g. \begin{equation} y(t) = A \, \sin(2\pi f+\phi) \, e^{-t/T} + \mathrm{noise}(\sigma) \end{equation} with $f \sim 1\,\mathrm{MHz}$ and $T \sim 10\,\mathrm{us}$.

Now I want to decide whether the signal is present in the noise and maybe get an idea of the amplitude. What is the best way to do this?

What I did in the past was taking 20 samples from the noisy system, uniformly spaced by 250 ns and then compute the FFT and square it. Then I look whether I see a peak at the expected frequency bin. However, this performs poor under heavy noise. I started to look into the book of Bretthorst (Bayesian Spectrum Analysis and Parameter Estimation) and there it is stated on page 20 that the FFT is not ideal if

• the number of samples is small
• the amplitude is decaying
• the frequency is low.

I tried to go down the road of a Bayesian approach as described in the book (starting at page 86). My initial trials to implement a Bayesian estimator from the Bretthorst book in Python were quite disappointing. I couldn't see an improved robustness against the noise.

Therefore, I am wondering: What is the best approach to decide whether I have a signal present? Computation time is not important to me (like no real-time stuff), and I think I have quite some pre-knowledge of my signal: I know that my frequency lies around $1\,\mathrm{MHz} \pm 200\,\mathrm{kHz}$. The phase $\phi$ is unknown, but I don't want to know it. For the decay time $T$, I know a lower bound, say $10\,\mathrm{us}$. In principle, I have a rough guess what the expected standard deviation $\sigma$ of the Gaussian or Poissonian noise is (at least an upper bound).

For the data acquisition, I can choose in principle also non-uniform sampling, but I can't increase the number of sampling points. Classical averaging, like taking multiple times 20 points of the same signal is not possible. I started to look into Wikipedia for Spectral density estimation but I feel overwhelmed by the number of techniques that are out there.

Answer 1

The problem you're running into seems to be that you have conflicting wants, specifically the fact that you want to operate at low SNR and use a small amount of samples. Integration gain needs to come in to play somewhere but limiting the amount of samples being used is working counter to that!

A couple of things you can try:

1. Leverage the known information you provided that the frequency lies around 1 MHz within a 400 kHz band. Bandpass filter $y(t)$ before processing it, this should give you about 10 dB of gain given that you're sampling rate is stated to be 4 MHz (samples uniformly spaced at 250 ns), while you only care about 400 kHz of that band (4 MHz / 400 kHz = 10).

2. Instead of looking at the FFT of $y(t)$, use the FFT of $y(t)$'s autocorrelation.

To see the improvement, plot your usual FFT, plot the FFT after filtering, and plot the FFT of the filtered signal's autocorrelation.