1
$\begingroup$

I have a time series, which I believe should have daily periodicity, the signal is sampled once every 15 minutes, which means that one period is 288 samples.

When I am trying to validate this result based on the FFT of the signal I get different results based on the number of points I sample from the FFT.

I am doing this in python, and first the number of points I sample the FFT is the same as the input array (900).

SEGMENT = np.fft.fft(segment-np.mean(segment))
freqs = np.arange(len(SEGMENT.T))/len(SEGMENT.T)
segment_abs = np.abs(SEGMENT)
fig = plt.figure(); ax = fig.add_subplot(111)
plt.plot(freqs,segment_abs.T,'o--')
plt.ylabel('Mangitude')
plt.xlim((0,0.05)); 
plt.xticks([0, 1/288], labels=['inf','Daily'])

ax.grid(); plt.show()

enter image description here

When I take the inverse of the X-axis I get that the top 2 frequencies are 245 and 332.

If I instead set the number of points to sample from FFT to 288, I get a clean peak which occurs at the 1/288 frequency.

enter image description here

I'm not sure how to interpret this, or how to think here.

If I want to determine if there is a period of 288 samples, which graph is better to look at?

Improve this question
$\endgroup$
2
  • $\begingroup$ Try an FFT length that is an integer multiple of 288. $\endgroup$
    – Hilmar
    Commented Sep 29, 2021 at 21:52
  • $\begingroup$ I'd suggest using the DTFT instead of the DFT (on the entire data) evaluated at 288. $\endgroup$
    – MBaz
    Commented Sep 29, 2021 at 22:12

1 Answer 1

Reset to default
-1
$\begingroup$

You need your period to fit an integer times into your DFT length, else you get leakage (not explaining that concept here, it's a very fundamental one and you can find plenty good material on it, here and in other places. Book: any textbook on digital signals).

But.

If I want to determine if there is a period of 288 samples, which graph is better to look at?

If you want to do that, doing a full FFT is the wrong method. Goertzel would be a better choice, or if you know there's only one strong harmonic signal in there, a parametric spectrum estimator (which can estimate the actual frequency instead of "checking" at a fixed raster)

If you want so "tolerance", a band-pass filter might be much lower in complexity (i.e. need fewer samples) than a Goertzel / FFT, too.

Improve this answer
$\endgroup$
6
  • $\begingroup$ For me the constraint is that it can be vectorized and fast. I am to do this for around 1000 time series at the same time. Do you have any suggestion on which method might suit best for this constraint? $\endgroup$
    – kspr
    Commented Sep 30, 2021 at 13:55
  • $\begingroup$ yes, that doesn't change my recommendations. $\endgroup$
    – Marcus Müller
    Commented Sep 30, 2021 at 13:58
  • $\begingroup$ (1000 Time series of a couple hundred samples each: nearly no computational load for modern PCs, at all, so unless you're planning on abandoning python to begin with, you're also barking up the wrong tree, performance-wise ;)) $\endgroup$
    – Marcus Müller
    Commented Sep 30, 2021 at 14:01
  • $\begingroup$ Thanks for the input! You don't happen to use python and know of any existing libraries for these methods you recommend? $\endgroup$
    – kspr
    Commented Sep 30, 2021 at 14:10
  • $\begingroup$ as said, the math here is so small, your overhead is in python. your bottleneck when processing so few samples is not the math as is - it's python being run on 1000 time series. Is your processing actually too slow? Because, honestly, I don't think it is. $\endgroup$
    – Marcus Müller
    Commented Sep 30, 2021 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.