2
$\begingroup$

My apologies for perhaps the stupidity of this question.

Presume that one has the 'frequency response' $Y_{data}(k)$ of a system and also has an estimated model $Y_{syn}(k)$ that fits the data.

How can I quantify the 'goodness of fit' of $Y_{syn}(k)$ to $Y_{data}(k)$ ? Notice that both curves are complex-valued at every instant k (i.e. frequency domain).

For example, assume that I have two estimated models: $Y_{syn}^a(k)$ and $Y_{syn}^b(k)$. My goal (besides visualizing the residual) is to evaluate in a objective way the fit of both models to $Y_{data}(k)$ (the original data set). Is there a single metric to do this?

$\endgroup$
1
$\begingroup$

You could use the $p$-norm of the complex error $E(k)=Y_{data}(k)-Y_{syn}(k)$:

$$\epsilon_p=\sqrt[p]{\sum_{k=1}^K|E(k)|^p}\tag{1}$$

The most common values for $p$ are $p=2$ (Euclidean norm, $l_2$-norm) and $p=\infty$ (maximum norm or Chebyshev norm):

$$\epsilon_2=\sqrt{\sum_{k=1}^K|E(k)|^2}\tag{2}$$ $$\epsilon_{\infty}=\max_{1\le k\le K}\{|E(k)|\}\tag{3}$$

It eventually depends on your application which error measure best reflects the quality of the approximation. Note that for the maximum norm $(3)$, a single index $k$ determines the value of the error measure $\epsilon_{\infty}$, whereas the error measure $\epsilon_2$ in $(2)$ depends on all values of the complex error $E(k)$, and it is proportional to the square root of the mean squared error.

$\endgroup$
  • 2
    $\begingroup$ And if you have some other knowledge about why you want to weight the error for different $k$, you can include a weighting: $$\epsilon^2_p=\sqrt[p]{\sum_{k=1}^K|w(k)E(k)|^p}$$ This might be useful if you have known values of $k$ for which you need close agreement (little error), so you choose $w$ for those $k$ values to be larger. $\endgroup$ – Peter K. May 19 '16 at 12:36
  • $\begingroup$ @PeterK. Would you consider $1/k$ a possible weighting function, since usually you plot the frequency on a log scale, but the steps in frequency are constant, such that you have a lot more points at high frequencies. And what about normalizing the error ($\left|\frac{E(k)}{Y(k)}\right|^p$), to avoid big impacts of small relative errors at high magnitude values of $Y(k)$ (which is also why you normally plot magnitude in decibel/logarithmic)? $\endgroup$ – fibonatic May 19 '16 at 19:56
  • $\begingroup$ @fibonatic : yes, that makes sense. Sometimes a relative error is what you need. Not sure I'm convinced by $1/k$, but it's certainly possible. $\endgroup$ – Peter K. May 19 '16 at 20:24
0
$\begingroup$

You could use common parameters, such as

  • sum-of-squared-residuals $\chi^2$ or the reduced version $\chi^2/N$ ($N$ is the degrees of freedom)
  • coefficient-of-determination $R^2$

Of course, you have to replace every square function by the square of the absolute value, i.e. $(x_i - \mu)^2 = (x_i - \mu)(x_i - \mu)^*$.

$\endgroup$
  • 1
    $\begingroup$ For the sake of clarity, I upvoted your answer as well, but I'm not allowed due to my low rep. $\endgroup$ – WaffleTeX May 19 '16 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.