1
$\begingroup$

I have a series of data containing 120,000 points. The mean of each N(=60) point is zero. I want to forecast the next 60 points using the ARMA model. My question is, specificaly, how to choose the appropriate model parameters (p and q) and the size of the training set.

Particularly, I run the ADF Test while providing the 100,000 points and getting the following results:

ADF Statistic: -52.49
p-value: 0.0
Critial Values:
   1%, -3.43
   5%, -2.86
   10%,-2.56

You can see the ACF and PACF and the absolute value of the FFT of the empirical ACF as follows. It seems that almost all lags are critical. enter image description here enter image description here

Then, I used a small subset of the data set (400 points) and applied the ADF test, and plotted ACF and PACF, the results are as follow: The last figure shows the absolute value of the FFT of the empirical ACF as follows.

ADF Statistic: -6.21
p-value: 5.38e-08
Critial Values:
   1%, -3.44
   5%, -2.86
   10%,-2.57

enter image description here enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ just to get a feeling: Could you run an FFT on your empirical ACF and display the absolute of the result? $\endgroup$
    – Marcus Müller
    May 1, 2021 at 12:56
  • $\begingroup$ @MarcusMüller Thanks for your time. You can see the results for both cases. I have obtained them by acf= sm.tsa.stattools.acf(data, nlags=100) plt.plot(np.abs(fft(acf))) $\endgroup$
    – JES0
    May 1, 2021 at 13:10
  • $\begingroup$ thanks! Huh, it looks like these spectra have a relatively sharp high-pass characteristic, which indeed means that a very significant length of ACF is needed to model your system. My guess is that this is due to some bias-erasing step in signal preprocessing that you could actually do without (and then erase the bias in a last step after all processing is done). Is that possible? $\endgroup$
    – Marcus Müller
    May 1, 2021 at 13:15
  • $\begingroup$ I am not much of a signal processing guy lol. what do you mean by sharp high-pass characteristic? "a very significant length of ACF is needed" > does it mean a large p and q is needed in ARMA model? "bias-erasing step"> I detrended every 60 points by its average as I am not interested in its mean but only its deviations. Would be possible to avoid the detrending step and applying it on ARMA's output @MarcusMüller $\endgroup$
    – JES0
    May 1, 2021 at 13:27
  • $\begingroup$ it probably would be possible, indeed! The detrending is what causes the sharp jump in your fft between frequency zero and the first bin; that is, in itself, ARMA behaviour which your prediction would need to model. If you skip that detrending, you'll very likely get better results. you can still detrend – at the very end. $\endgroup$
    – Marcus Müller
    May 1, 2021 at 13:48

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.