# How to smoothen signal with missing values before differentiation?

I want to differentiate a noisy signal with many, randomly located, missing data values. Which smoothing techniques should be applied before differentiating the signal?

I have a velocity signal at 200 Hz, but due to wireless transmission about half the data values are missing (NaN), up to 8 in a row (40 ms between data points). I would like to get the derivative signal (acceleration), but without smoothening, it is too noisy. I'm mostly interested in a rough (!) estimation of maximum acceleration. What is the best techniques to handle the missing data? Linear interpolation between known data points?

• A Kalman Filter can be formulated on non uniformly spaced samples. – Stanley Pawlukiewicz Mar 28 '18 at 3:54
• Good idea! Thanks! A bit more effort then I was looking for but this seems like a good solution. – user1323995 Mar 28 '18 at 8:44

## 1 Answer

The best tool for this job is normalized convolution. It can deal with missing samples as well as uncertainty.

The paper describing the method is "Normalized and Differential Convolution -- Methods for Interpolation and Filtering of Incomplete and Uncertain Data" by Hans Knutsson Carl-Fredrik Westin. There is a PDF on Semantic Scholar.

Normalized convolution basically takes a convolution with a Gaussian of the data, with 0 for the missing samples, and divides sample-wise with the result of the same convolution on a weight signal, which is 1 for known samples and 0 for the samples you set to zero. More formally:

$$g = (( f m ) * h ) / (m * h)$$

(With $h$ the convolution kernel and $m$ the weight signal). You can derive the analytic derivative of that equation, leading to a derivative operator that is robust to missing samples. See the paper for more details.