I have a noisy signal with frequency between 1/3 and 1/5 of the sampling window.
DFT has a very low resolution for low frequency components, what is the best way to find the frequence?
Check out this article by Eric Jacobsen and Peter Kootsookos: http://www.ingelec.uns.edu.ar/pds2803/Materiales/Articulos/AnalisisFrecuencial/04205098.pdf
I had some success using this method for doing "sub-bin" frequency estimation when only one sinusoid is present.
Have you tried just zero-padding?
Suppose I have:
SINE = sin(2*%pi*0.001923492369*[0:1023] + rand(1)2%pi);
And I zero-pad it:
SINE_ZERO_PADDED = [SINE zeros(1,10240)];
Then I can take the FFT of both:
where the blue plot is the absolute value of the FFT of the original, and the red (dashed) plot is the absolute value of the FFT of the zero-padded signal.