How to evaluate performance of a model after estimating ARMA/MA/AR parameters for any process x(n)?

How to regenerate back a process after estimating average parameters? what kind of performance measure can be used to evaluate the corresponding model?

  • $\begingroup$ Welcome to DSP.SE! That's really quite a broad question. It really depends on why you're estimating the model: how do you quantify "good performance"? Or, another way of looking at it: what procedure did you use to estimate the parameters? That will have a figure of merit. What does that figure of merit tell you? $\endgroup$ – Peter K. Sep 3 '15 at 11:55
  • $\begingroup$ @PeterK.: Sir I am using Levinson Durbin, Durbin method and a cepstrum based method (called as LAST) for AR, MA and ARMA parameter estimation. After fitting a WSS process using all three AR(p), ARMA(p,q) and MA(q) models, how to decide which model among these is best for that particular process? (assuming 'p' and 'q' are any two integers) $\endgroup$ – Pranav Deshpande Sep 4 '15 at 16:36
  • 1
    $\begingroup$ So, if the three procedures are applied to the same time series, look at the innovations of all of them. If they are all white, then the one with the lowest innovations variance might be a good candidate as the "best". $\endgroup$ – Peter K. Sep 4 '15 at 16:42
  • $\begingroup$ PeterK is quite correct and should have posted it as a solution; IMHO. An additional note is that residuals should be examined to be normal or log-normal. If not: understand why. If so: double check :) Usually a normal probability graph together with the error boundary curves is sufficient to get you close to "noise temperature"; i.e. the point where further effort is wasted;and possibly not measuring data but noise. That is fitting numbers that will never occur again; whereas you want get rid of them. $\endgroup$ – rrogers Sep 9 '15 at 13:20
  • $\begingroup$ I think the performance metric is application dependent. The innovation approach essentially measures how well you simulate future values. If your problem has thresholds, a different approach would be to categorically divided the forecast and data into within/exceed thresholds for a period of time, and then other metrics, such as the frequency of forecasting within threshold but experiencing values exceeding the thresholds. $\endgroup$ – user3969377 Dec 31 '15 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.