Currently I am working in a spectrometer with a stage of decimation previous the fft. In particular I am decimating with a factor of 16 in two stages of 4 each one. I attach a diagram of the model (I'm using fixed representation by the way)
The problem its when i put certain frequencies as inputs the noise floor in the spectrum begin to oscillate. I had used windows believing that it is a sort of leakage problem, but still got the same behavior for other windows.
Here it is a video of one test I made, one channel has the signal windowed and the other no. https://www.youtube.com/watch?v=xhIDV4CRLoQ
The solution I have encountered it's truncating the output of the fft and that mitigates the oscillation and maintain the noise floor stable..but that affects my dynamical range.
I am sure the problem is in the decimation stage, but i searched and there are not info about it :(
I design the antialliasing filters for the decimation using the fdatool of matlab using the least square method and they have the following parameters:
wpass:0.20, w_stop:0.3, order: 111
wpass:0.25, w_stop:0.27 order: 159