# Causal Noise Free 1st Order Derivative in Discrete Domain

I need to have causal noise free first order derivation (Derivative). Now I am using a simple finite differences formula:

$$\frac{ x \left( n \right) - x \left( n - 1 \right) }{ {T}_{s} }$$

The filter above, being High Pass, is super sensitive to noise. Do we have a noise free or noise reduced first order differentiation which does not need more than three point for calculation?

• Why not lowpass filter afterwards? Feb 11 '16 at 21:13
• Have a look at Savitzky-Golay filters. Feb 11 '16 at 21:25
• Thank you Matt, I have took a look at SG filter. It works perfectly with low number of points. The problem is the derivation technique should be causal. Meaning it has to take a look at the previous points and predict the next points. The SG filter, low pass filter, moving average, ... delay the waveform.
– aghd
Feb 12 '16 at 13:29
– Royi
Sep 22 at 9:20

Well, Since basically the Derivative Operation is Linear Filter you can chose your own optimal trade off between Noise Sensitivity and Bandwidth.

If you look at Finite Differences Coefficients page at Wikipedia you can see you can chose higher "Accuracy" filters for 1st Derivative.
You can also chose Forward / Central / Backward method.

• The longer filters as listed are actually more sensitive to noise, using as performance figure the root-mean-square gain of white noise input, which equals the square root of the sum of the coefficients. In the central finite difference case the coefficients of the longer filters more accurately approximate samples of the derivative of a sinc function. In that sense the bandwidth stays the same. Jul 3 '17 at 11:41
• @OlliNiemitalo, I am not sure about your assertion. One intuitive way to see it is using a Linear Function. Imagine you have a function $f \left( {x}_{i} \right) = a {x}_{i} + b$ for $i = 1, 2, \ldots, N$. Estimating the derivative which equals to estimating the parameters $a$ is better with more coefficients used. Now, the trick is the data is stationary (Namely the derivative is the same). But the noise, which is the thing you talked about, is better with more samples. This is because looking at more samples in that context is like looking at lower bandwidth.
– Royi
Sep 22 at 9:18
• The comments aren't best to describe what I want to say. But I wanted to say it is not all in the sum of squares of coefficients. Longer filters for the correct model means better resistance to noise as well.
– Royi
Sep 22 at 9:19

You can use differentiator filter that acts as a differentiator in the band of interest, and as an attenuator at all other frequencies, effectively removing high frequency noise. Take a look at the matlab example

If there was no noise in the input, then the causal filter:

$$y[k] = \frac{\frac{3}{2}x[k] - 2x[k-1] + \frac{1}{2}x[k-2]}{T_s}$$

would give the asymptotically optimal 3-sample linear approximation of the first derivative in the limit of an infinite sampling frequency assuming zero input noise. The asymptotic error as sampling period $$T_s\to0$$ is a good approximation of the true error if your have a very large sampling frequency compared to the frequencies of the signal. Because the magnitudes of the coefficients are so large, this filter does not work very well with noisy input. The root-mean-square (RMS) gain of white noise input is for this filter $$\sqrt{(\frac{3}{2})^2 + (-2)^2 + (\frac{1}{2})^2} \approx 2.549509756$$, compared to $$\sqrt{1^2 + (-1)^2} \approx 1.414213562$$ for your filter.

Recycling the symbol $$x$$ to be the argument of an underlying continuous function $$f(x)$$, the coefficients $$(c_0, c_1, c_2)$$ were calculated by making the error in the derivative approximation calculated at $$x = x_0$$ vanish in the limit $$T_s\to0$$:

$$\lim_{T_s\to0}\left(\frac{c_0f(x_0) + c_1f(x_0 - T_s) + c_2f(x_0 - 2T_s)}{T_s} - f'(x_0)\right) = 0\\ \Rightarrow c_0 = -c_1 - c_2 \text{ and } c1 = - 2c_2 - 1,$$

and by making first-order asymptotic estimate of the error vanish:

$$T_s\lim_{T_s\to0}\left(\frac{\frac{(c_2 + 1)f(x_0) + (-2c_2-1)f(x_0 - T_s) + c_2f(x_0 - 2T_s)}{T_s} - f'(x_0)}{T_s}\right) = 0\\ \Rightarrow c_2 = \frac{1}{2}.$$

What remains is second-order asymptotic estimate of the error:

$$T_s^2\lim_{T_s\to0}\left(\frac{\frac{\frac{3}{2}f(x_0) - 2f(x_0 - T_s) + \frac{1}{2}f(x_0 - 2T_s)}{T_s} - f'(x_0)}{T_s^2}\right)\\ = -\frac{1}{3}T_s^2f^{(3)}(x_0),$$

where $$f^{(n)}(x)$$ denotes the $$n\text{th}$$ derivative of the underlying continuous function. An alternative way to look at this is that this is the third and the first non-zero term of the Taylor series (or MacLaurin series) of the derivative approximation error about $$T_s = 0$$.

In contrast, your 2-point filter with coefficients $$(1, -1)$$ has zero limiting error as $$T_s\to0$$ and possibly non-zero first-order asymptotic estimate of the error in the limit $$T_s\to0$$:

$$T_s\lim_{T_s\to0}\left(\frac{\frac{f(x_0) - f(x_0 - T_s)}{T_s} - f'(x_0)}{T_s}\right)\\ = - \frac{1}{2}T_sf^{(2)}(x_0).$$

As $$T_s\to0,$$ the first-order asymptotic estimate of the error tends to zero slower than the second-order one because of the different exponent of $$T_s$$, so the 3-point filter will for some non-zero $$T_s$$ give less error than the 2-point filter, unless the function happens to have enough vanishing derivatives at $$x_0$$ in which case the approximation would be good enough already with the 2-point filter.

The presented approach would work for longer filters too. It appears to give the same coefficients as the also Taylor series based finite difference method as linked to by Royi.

The requirement for causality can be dropped, giving a better 3-point (well, the middle coefficient is zero) asymptotically optimal filter in the limit $$T_s\to0$$:

$$y[k] = \frac{\frac{1}{2}x[k+1]-\frac{1}{2}x[k-1]}{T_s},$$

with second-order asymptotic estimate of the error:

$$\frac{1}{6}T_s^2f^{(3)}(x_0),$$

and a remarkably low RMS gain of white noise input of $$\sqrt{(\frac{1}{2})^2 + (-\frac{1}{2})^2} = 0.7071067811$$. If you shift the filter in time as:

$$y[k] = \frac{\frac{1}{2}x[k]-\frac{1}{2}x[k-2]}{T_s},$$

then the limiting error at $$T\to0$$ will still be zero and the first-order asymptotic estimate of the error will be:

$$-T_sf^{(2)}(x_0).$$

We have seen earlier that for a 3-point causal filter the coefficients must be $$(c_2+1, -2c_2-1, c_2)$$ for the limiting error to vanish as $$T_s\to0$$. The white noise RMS gain $$\sqrt{(c_2+1)^2 + (-2c_2-1)^2 + c_2^2}$$ is minimized by the above filter with $$c_2 = -\frac{1}{2}$$. The absolute value of the error coefficient $$b$$ in the first-order asymptotic estimate of the error $$bT_sf^{(2)}(x_0)$$ is $$1/2$$ for your filter. Now, the question is, is there a compromise filter that can improve one of the performance figures without making the other worse? Unfortunately, the answer is no: Figure 1. white noise (WN) RMS gain and the absolute value $$|b|$$ of the coefficient in the first-order asymptotic estimate of the error as function of $$c_2$$ for the compromise filter with coefficients $$(c_2+1, -2c_2-1, c_2).$$ Your filter with coefficients $$(1, -1)$$ or $$(1, -1, 0)$$ is at $$c_2 = 0,$$ marked by dots on the curves.

If you want to improve the noise figure, you must allow a larger error figure, by picking an intermediate value of $$c_2$$ between $$-\frac{1}{2}$$ and $$0$$ in the compromise filter:

$$y[k] = \frac{(c_2+1)x[k] + (-2c_2-1)x[k-1] + c_2x[k-2]}{T_s},$$

with first-order asymptotic estimate of the error:

$$\frac{2c_2 - 1}{2}T_sf^{(2)}(x_0).$$

The second derivative $$f^{(2)}(x)$$ is related both to the bandwidth and to the amplitude of your signal.

If your "signal" is linear, then its second derivative is zero, and you can minimize white noise RMS gain without an error penalty. This gives as optimal the three-point filter already discussed earlier, with coefficients $$(\frac{1}{2}, 0, -\frac{1}{2})$$:

$$y[k] = \frac{\frac{1}{2}x[k] -\frac{1}{2} x[k-2]}{T_s}.$$

This should halve the white noise RMS gain compared to your filter. Note that for linear input the first derivative is constant, so the coefficients are the same as for the Savitzky–Golay differentiation filter $$y[k] = \frac{\frac{1}{2}x[k+1] -\frac{1}{2} x[k-1]}{T_s}$$ (also discussed earlier) which has equivalent performance to the causal filter for the linear signal model.

You didn't say how you got your x(n) values, so Automatic Differentiation might be a solution,

https://en.wikipedia.org/wiki/Automatic_differentiation

and

http://www.autodiff.org/?module=Tools&language=MATLAB

These are not based on finite differences.