0
$\begingroup$

I want to differentiate a noisy signal with many, randomly located, missing data values. Which smoothing techniques should be applied before differentiating the signal?

I have a velocity signal at 200 Hz, but due to wireless transmission about half the data values are missing (NaN), up to 8 in a row (40 ms between data points). I would like to get the derivative signal (acceleration), but without smoothening, it is too noisy. I'm mostly interested in a rough (!) estimation of maximum acceleration. What is the best techniques to handle the missing data? Linear interpolation between known data points?

$\endgroup$
  • $\begingroup$ A Kalman Filter can be formulated on non uniformly spaced samples. $\endgroup$ – Stanley Pawlukiewicz Mar 28 '18 at 3:54
  • $\begingroup$ Good idea! Thanks! A bit more effort then I was looking for but this seems like a good solution. $\endgroup$ – user1323995 Mar 28 '18 at 8:44
1
$\begingroup$

The best tool for this job is normalized convolution. It can deal with missing samples as well as uncertainty.

The paper describing the method is "Normalized and Differential Convolution -- Methods for Interpolation and Filtering of Incomplete and Uncertain Data" by Hans Knutsson Carl-Fredrik Westin. There is a PDF on Semantic Scholar.

Normalized convolution basically takes a convolution with a Gaussian of the data, with 0 for the missing samples, and divides sample-wise with the result of the same convolution on a weight signal, which is 1 for known samples and 0 for the samples you set to zero. More formally:

$$ g = (( f m ) * h ) / (m * h) $$

(With $h$ the convolution kernel and $m$ the weight signal). You can derive the analytic derivative of that equation, leading to a derivative operator that is robust to missing samples. See the paper for more details.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.