3
$\begingroup$

I have a pretty simple question which i should have been able to answer. Just wanted to check if people here have a better solution to it.

I am trying to establish that FFT as a frequency estimator (imagine a single sinusoid + AWGN model) is unbiased. I guess we can agree that the answer to that depends on the number of grid points on the frequency axis or put in a different way it depends on how many zeros you pad to your signal. And no matter what, you will always be limited by your grid resolution. There are many papers out there which sort of interpolate using samples around the detected peak to refine the estimate and that pretty much is unbiased. This question can then be extended to other sort of correlation based estimators such as range estimation for a radar wavefrom etc.

I have a similar setting that i am working on, skipping the details for brevity but the signal model and algorithm are essentially what i described above. I plotted the resulting errors and they do come out to be unbiased. I was just wondering if i can somehow prove mathematically that the estimator is indeed unbiased. The reason i want to do that is because i eventually want to compare with the CRB.

If any papers/lectures or ideas come to mind, do share them. Thanks.

Improve this question
$\endgroup$
  • $\begingroup$ CRB : You mean Cramer-Rao lower bound? $\endgroup$ – Ben Aug 10 '20 at 16:48
  • $\begingroup$ @CedronDawg has done considerable work studying this specifically and should be able to help you (tagging him to see this) $\endgroup$ – Dan Boschen Aug 10 '20 at 17:09
  • $\begingroup$ @Ben -- yes that's what i meant. Thanks for the clarification. $\endgroup$ – Khurram Aug 10 '20 at 18:38
  • $\begingroup$ @DanBoschen -- Thanks. Hopefully Dan will have something for me to go on. $\endgroup$ – Khurram Aug 10 '20 at 18:39
  • $\begingroup$ Unbiased is not directly associated to the Cramer Rao lower bound. An estimator could be unbiasedly but not acheive Cramer Rao lower bound. Although it is sure that Cramer Rao bound, if reached will be by an unbiased estimator $\endgroup$ – Dsp guy sam Aug 10 '20 at 19:02
1
$\begingroup$

A had a long email discussion about the topic of bias in frequency formulas, aka estimators with a really sharp Grad student a while back. It makes me hesitant to speak authoritatively on statistical matters.

I don't like the term estimator as it is too broad.

Formulas come in two basic flavors, either they are exact or they are not. Exact means mathematically exact. So, if you feed it the DFT of a noiseless pure tone (real or complex, different formulas for each), the calculation will result in an answer limited only by the precision of the calculation.

Other formulas are approximations. Approximations are often times cheaper to calculate and good enough for the job. In heavier noise, the precision advantage of a more precise formula may be rendered useless, even a disadvantage. Approximation formulas are inherently biased. Run a set of tones through them and maybe, like a broken clock, it will appear unbiased for certain tones.

Now run add your run-of-the-mill AWGN to the pure tone signal and test the frequency formulas. Does the Gaussian shape pass through unskewed and still centered?

The (excellent, Thanks Julien) paper I cited in the comment didn't take quite that approach.

I would be quite interested in another, up to date, comparison with emphasis on CRLB calculation (especially on the real tone case) of all my formulas against the known standards (Start with the cited list, I think he missed Macleods.)

Here is another similar page:

http://www.ericjacobsen.org/fe2/fe2.htm

If you do endeavor to do a comparitive analysis, and there no better way to learn the alternatives, and post it somewhere, please cite it in a comment on this answer.

Frequency on bin (which seems to be what you are asking about in terms of a native bin) cases are sort of a different beast than non-whole cycle per frame tones.

The technique cited in my articles can be extended to any arbitrary number of bins.

Improve this answer
$\endgroup$
  • $\begingroup$ I am not sure you cited a paper/article with Julien as one of the authors. Anyway, your articles and the link in your reply are just different variants of frequency estimation. And i am sure they are all useful and provide a lot of insight. My original question, though, was directed in a different direction as explained in my earlier comments. As for comparing all of them to the CRB, that's not my focus right now. But if i do end up doing something about that, i will for sure link it here. $\endgroup$ – Khurram Aug 11 '20 at 16:15
  • $\begingroup$ Also, this is just language, but i think i should still clarify it. You said "CRB calculation of all my formulas". I don't think that's a very precise statement. CRB doesn't depend on what formula (or algorithm/estimator) one is using. It depends on your signal model which for the purpose of all of our discussion is a sinusoid + AWGN. Just a clarification for other people who end up reading this. $\endgroup$ – Khurram Aug 11 '20 at 16:17
  • $\begingroup$ @Khurram I guess that was sloppily said. Julien's frame of reference was the CRLB. I'm not sure if it is a bound or a limit. It seems to me your first step is to understand CR thoroughly and when you do, if not already, you are ahead of me. When I presented my testing on comp.dsp, it got sidelined by endless discussions of "proper noise" and "testing techinques". That ain't my playground. Whether a formula is biased, that is skews a probability distribution sideways, seems to me to be a different issue than if it is as precise as can be. I'd rather have a skinny slightly off center... $\endgroup$ – Cedron Dawg Aug 11 '20 at 17:18
  • $\begingroup$ ....than a wide centered one. I always found it easier to numerically test and study the results. The theory has to match that anyway. $\endgroup$ – Cedron Dawg Aug 11 '20 at 17:18
  • $\begingroup$ CRB is a lower bound on the variance of your estimator error. Any unbiased estimator can't outperform it. A biased one can. This brings us to what you said: In some cases it might be better to have a skinny off center error distribution rather than a wide centered one. That's just a biased estimator. $\endgroup$ – Khurram Aug 12 '20 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.