3
$\begingroup$

Apologies in advance for what I imagine is a simple question, however I can't work out how to do this.

I have two signals of the same type but with different sensor types, the figure below will give you an idea of some of the data I've got:

enter image description here

I very simply want to calculate the average percentage difference between the two signals. Methods I've attempted so far have calculated outrageous results, in the order of ~200% difference. My attempt is below: 

  for i = 1:length(data)
      per(i) = abs((var1(i)-var2(i))/((var1(i)+var2(i))/2)) * 100;
  end

I believe the problem in this method is that I'm only calculating the difference between the two immediate points in question (i) and not on a grand scale of all the data.

Any help would be greatly appreciated.

|improve this question
$\endgroup$
  • $\begingroup$ Try searching for "normalized cross correlation". It might also be called the "sample correlation coefficient". The result is a value in the range $[-1,1]$ indicating the similarity in a vector-space sense between two signals. $\endgroup$ – Jason R Feb 10 '14 at 18:36
  • $\begingroup$ A major problem with the way you're calculating it is the fact that the expression per(i) becomes singular whenever var1(i)+var2(i) becomes zero, so you could get nonsensical catastrophic numerical errors tending towards infinity if the signal "accidentally" satisfies this equality. $\endgroup$ – DumpsterDoofus Feb 10 '14 at 22:05
5
$\begingroup$

At the least you should normalize it by the length of the sequence. But what you're trying to do is a pretty standard thing. I'm not sure what percentage difference really means. What you should probably be using is something like the root mean squared error:

$$RMSE=\sqrt{\frac{1}{N}\sum_{n=0}^{N-1}(x(n)-y(n))^2}$$

where $x(n)$ and $y(n)$ are your two sequences.

|improve this answer
$\endgroup$
  • $\begingroup$ This seems like the right way to go. Thank you. $\endgroup$ – ritchie888 Feb 11 '14 at 13:52
3
$\begingroup$

In addition to the RMSE interpretation already given, another visually obvious way to interpret "percentage difference" is via the standard Euclidean norm. The two signals, assuming they're sampled uniformly over the interval, can be thought of as two different vectors in $n$-dimensional space, and they're pointing in very similar directions. Thus, one can interpret the percentage difference as $$d=\frac{|v_1-v_2|^2}{|v_1||v_2|},$$ which can be implemented by 

dot(var1-var2, var1-var2)/sqrt(dot(var1,var1)*dot(var2,var2))
|improve this answer
$\endgroup$
2
$\begingroup$

You are calculating the relative error between two curves, when you curves have the opposite sign, the difference between the two curves may be even larger than the amplitude of the curve. That's probably the reason why your mean(per) reaches 200.

Like @Eric said, RMSE makes more sense, the Matlab implementation is 

RMSE = sqrt(mean((var1-var2).^2));
|improve this answer
$\endgroup$
  • $\begingroup$ Thank you for your input and the code snippet. Very useful. $\endgroup$ – ritchie888 Feb 11 '14 at 13:52
0
$\begingroup$

I had a similar situation as yours and I chose the mean absolute percentage error http://en.wikipedia.org/wiki/Mean_absolute_percentage_error.

$$mape=\dfrac{1}{n}\sum{\left|\dfrac{m_i-s_i}{m_i}\right|}$$ where $m_i$ and $s_i$ are the measured and simulated values resp.

You can implement this in R as mape = mean((m+s)/m) where m and s are arrays with the values.

If you want to see the mape at a specific point just use $$mape_i = \left|\dfrac{m_i-s_i}{m_i}\right|$$

I like this because you can analyse how the error changes overtime.

I used both errors measures. I used the later because, in my problem, the domain was a parameter of my model, and I wanted to see how good my model was for different values of the parameter.

Disclaimer: I did computer science, thus I'm not mathematician nor statistician...

|improve this answer
$\endgroup$
  • $\begingroup$ Why not to at least put the equation here, instead of linking to the Wikipedia? $\endgroup$ – jojek♦ Jun 11 '15 at 8:13
  • $\begingroup$ because I don't know the markup for formulas and in the wikipedia the formula is already clear. $\endgroup$ – jbarrameda Jun 11 '15 at 17:56
  • $\begingroup$ Here is an example tutorial $\endgroup$ – jojek♦ Jun 11 '15 at 18:11
  • $\begingroup$ Of course it is. $\endgroup$ – jojek♦ Jun 11 '15 at 18:54
  • $\begingroup$ Voila. Just edited the answer. Hope it helps (and maybe we get feedback from a mathematician). Thanks. $\endgroup$ – jbarrameda Jun 11 '15 at 19:01
-3
$\begingroup$

You can use this command:

plot(T,100*(var1-var2))

Hope it might solve your problem although you don't need a plot command.

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.