There are many related questions posted about estimating sine wave parameters, and this one is closest: Measuring amplitude of a pure sine wave of known frequency close to the noise floor, but none consider the specific case of providing the optimum strategy when signal is near the noise with unknown frequency in the presence of non-stationary noise (such as phase noise).
My question is as follows:
Given a signal capture consisting of a real sinusoid of unknown frequency (other than not being near DC or Nyquist) and noise, what is the optimum strategy to make the best estimate of the sinusoid's amplitude under low SNR conditions?
Please provide sufficient detail of approach for me to validate against a test waveform and from that I will select the winner (For a consistent metric, I will test the algorithm multiple times with independent runs of the test waveform using a sinusoid at -45 dBm with phase noise as given in plot below and thermal noise at -76 dBm/Hz, and choosing the solution with lowest rms error from true amplitude).
The test constraints are detailed below:
The frequency $\omega$ is $\pi/8 \le \omega \le 3\pi/8$ radians/sample (This is just to avoid further complications when we get near DC or Nyquist).
The capture can be as long as necessary to get the best estimate, including captures for parameter adjustment (sequentially, not concurrently).
The only amplitude variation is from white noise (thermal noise floor dominated by local receiver, so additive to the sinusoid as received).
In addition to white thermal noise, the sinusoid has phase noise (collectively from the transmitter, receiver and sampling clock).
The sinusoid is received in a low SNR condition. The sinusoid itself is not purposely changing with time (no additional modulation) and there are no other interference signals.
We have the convenience of acquiring a stronger SNR copy from the representative noise process (as I reveal below for my test case), but not concurrent to the measurement (assume the stronger measurement case is done much earlier in time and at an arbitrarily different frequency - meaning we cannot use that in any way to estimate the frequency of the tone for the actual measurement capture).
There is only one capture that can be done concurrently (one ADC) and we have no way of getting any other additional information about the signal or measurement other than the results of any one capture alone.
The channel is not changing with time so only has effected the attenuation as received (to keep further complications with channel effects out of this).
Below is a plot of the phase noise spectrum for the test case I will use, as we may have measured it under high SNR conditions. Specifically this shows the power due to phase fluctuations of the sine wave due to all noise sources (local oscillators in transmitter and receiver and sampling clock). If it helps to have numbers, this shows a $1/f^2$ in power (random walk in phase) for 1 KHz and below (-35 dBc/Hz at 100 Hz), and $1/f$ in power (pink noise) from 1 KHz and above. The plot shown here intersects with white noise at 100 KHz, but the white noise contribution relative to received signal would vary with the SNR condition.