1
$\begingroup$

I ran an FFT on real financial monthly time series data. If I plot the FFT frequency domain output on the interval $[0, f_s)$, the dominant frequency pair peaks occur at $f_{A1}$ $\approx$ $0.02 \ month^{-1}$ and $f_{A2} \approx 0.98 \ month^{-1}$. If I plot the FFT frequency domain output on the interval $[-f_s/2, f_s/2)$ instead, then they occur at $f_{B1} \approx -0.48 \ month^{-1}$ and $f_{B2} \approx 0.48 \ month^{-1}$.

Because the original time domain data is real with no imaginary component, I think that it would make sense to plot the FFT frequency domain data on the interval $[0, f_s/2)$ and ignore the negative frequencies (as discussed in a related question I posted: "Binning and Frequency for FFT on Financial Time Series Data"). However, if I ignore the negative frequencies and plot only $[0, f_s/2)$, I think the FFT might not reflect the original time series as I'll explain below.

When I eyeball the original time series data, I see what looks like a predominant cycle period that repeats roughly every $T_{eyeball} \approx 36 \ months$, give or take. I thought that this should roughly correspond to the peak frequencies in the frequency domain, which should be about $f_{eyeball} \approx 1/T_{eyeball} = 1/(36 month) = 0.028 \ month^{-1}$. $f_{eyeball}$ is close to $f_{A1} \approx 0.02 \ month^{-1}$, the first peak frequency if I plot my frequency domain data on the interval $[0, f_s)$ as described in the first paragraph above. $f_{eyeball}$ is not close to the peak frequencies $f_{B1}$ and $f_{B2}$ that result from plotting the frequency domain on the interval $[-f_s/2, f_s/2)$. So, while it makes conceptual sense to me to plot the FFT on the $[0, f_s/2)$ interval because my time series data is real, that seems to be at odds with the nature of the time series.

If I were to plot on $[0, f_s)$ thus capturing the whole spectrum, it would seem like I'm double-counting frequencies because of the symmetry of FFT. Could the solution be to take the data on $[0, f_s/2)$? This would capture the peak frequency that seems to fit the data, but it seems like an arbitrary choice.

Could it be possible that this is an artifact of how scipy.fft organizes positive an negative frequencies? The scipy.fft` documentation states:

"For N even, the elements $y[1] ... y[N/2 - 1]$ contain the positive-frequency terms, and the elements $y[N/2] ... y[N - 1]$ contain the negative-frequency terms, in order of decreasingly negative frequency. For N odd, the elements $y[1] ... y[(N - 1)/2]$ contain the positive-frequency terms, and the elements $y[(N+1)/2] ... y[N - 1]$ contain the negative-frequency terms, in order of decreasingly negative frequency."

If I'm understanding this correctly, this would mean that scipy.fft bins positive frequencies on the $[0, f_s/2)$ interval and the negative frequencies on the interval $[f_s/2, f_s)$. This would mean that choosing $[0, f_s/2)$ is the right way to go for me and that would be consistent with the correct peak frequency $f_{A1}$ $\approx$ $0.02 \ month^{-1} \approx f_{eyeball} \approx 0.028 \ month^{-1}$

Or could it be something else that I'm missing?

$\endgroup$
0

1 Answer 1

0
$\begingroup$

I appreciate your efforts in describing your problem in such details. I almost feel bad for the time it must have taken you! But it looks like by writing your question, you got to the answer yourself.

Yes, cut your fft output (of length $\tt{Nfft}$) in half (only keep indexes $0$ to $\tt{Nfft}/2−1$ if $\tt{Nfft}$ is even, $\frac{\tt{Nfft}−1}{2}$ if $\tt{Nfft}$ odd), which discards the negative frequencies and keeps only the positive ones, i.e. $\texttt{DC}$ to $f_s/2$.

You seem to also be running in resolution issues. The reason is that (judging by your previous question), the resolution is $$d_f = \frac{1}{\Delta_t \times N} = 0.0032\, \,\texttt{month}^{-1}$$ where $N$ is the length of your data. That means that each bin $k$ contains the information at frequency $0.0032k \,\,\tt{month}^{-1}$

It looks like you zero-padded your waveform to the next available power of $2$, i.e. $\tt{Nfft} = 512$, giving you a precision of $1/512 = 0.02 \,\, \tt{month}^{-1}$.

You can zero-pad further by increasing $\tt{Nfft}$, which interpolates the spectrum further. For example, using an FFT length $\tt{Nfft}=1024$, will give you $0.01\,\,\tt{month}^{-1}$ precision. You should then see a peak at $0.03\,\,\tt{month}^{-1}$, closer to what you eye-balled as being $0.028\,\,\tt{month}^{-1}$

$\endgroup$
4
  • $\begingroup$ Yes, writing it out did help me get to the answer. Nevertheless, I appreciate your not only verifying my conclusion but also commenting on resolution and padding. To your point about padding, I did not pad my data, so could it be possible that scipy.fft did that automatically? Part of my issue is that I'm learning the nuances of that package. $\endgroup$ Commented Apr 8 at 23:02
  • $\begingroup$ hmm. If you didn't specify a padding length, your FFT output should be the same length as your data, how did you construct your frequency vector? did you use fftfreq? How long is your input data? $\endgroup$
    – Jdip
    Commented Apr 8 at 23:48
  • $\begingroup$ My data has 312 points. The FFT output has 312 points too. I didn't create a frequency vector. I plotted the frequencies in Excel and found the maximum frequency, which was the only one I was looking for. Using fftfreq to make a proper vector sounds like a good idea. $\endgroup$ Commented Apr 9 at 1:11
  • $\begingroup$ ...when I say "maximum frequency", I really mean the most prevalent frequency--the one where the frequency domain spectrum peaks. $\endgroup$ Commented Apr 9 at 1:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.