# What criteria should be used for smoothing

Actually I have some measurement which I want to get rid of noise I want to use different filter techniques but I am wondering what criteria I should use to check if I am removing noise or not or if my selected parameter is optimal.

My measurement is mass spectrometry data.

• Hi! you shall better consider revising your question, by giving clear and detailed information about your setup, before it's probably put on hold by moderators... – Fat32 May 20 '16 at 20:18
• @Fat32 I gave the info about the type of data I have – nik May 20 '16 at 20:26
• Ok. It's very clear now. Let's hope you will have answers. – Fat32 May 20 '16 at 21:16
• @nik high-res MS? – Laurent Duval May 23 '16 at 19:45

You want to look at the innovation sequence; the difference between the filtered value (the estimate or forecast) and the measured value. If you have a perfect filter that removes the noise, the innovation sequence should be white noise (Dirac impulse autocorrelation).

https://en.wikipedia.org/wiki/Innovation_(signal_processing)

Here is some MATLAB code that illustrate these properties for some white sequences.

N = 1e5;
n1 = -1 + 2*rand(1,N); % uniform noise
n2 = n1 + -1 + 2*rand(1,N); % triangular noise
n3 = randn(1,N); % Gaussian noise

[c1,l1] = xcorr(n1); [c2,l2] = xcorr(n2); [c3,l3] = xcorr(n3);
figure(1), plot(l1,c1,l2,c2,l3,c3) # autocorrelation (Dirac delta)
figure(2), clf, hold on
pwelch(n1,kaiser(N/1000,10),N/2000) # power spectral density (constant)
pwelch(n2,kaiser(N/1000,10),N/2000)
pwelch(n3,kaiser(N/1000,10),N/2000)

• how can I define white noise in my data ? do you have any example in R ? or matlab or even python ? – nik May 21 '16 at 6:43
• White noise is a definition; it means that the samples are un-correlated, which means that there is no memory or dependence between them -- hence the autocorrelation of a white noise sequence is the Dirac delta, which means that the all the samples are only correlated with themselves. – Arnfinn May 21 '16 at 8:16

Kalman filtering techniques require that you can model how your signal (the thing you're interested in) evolves with time and how the noise corrupts it.

The signal model is usually of the form: $$\underline{x}_{k+1} = \mathbf{A} \underline{x}_k +\mathbf{ B} \underline{w}_k\\ z_k = \mathbf{C} \underline{x}_k + \underline{v}_k$$ where

• $\underline{x}_k$ is your system state at time $k$,
• $\underline{w}_k$ is the process noise which describes the statistics and how energetic your state changes are,
• $\underline{v}_k$ is the measurement noise which describes how your measurements are corrupted, and
• $\mathbf{A},\mathbf{B},\mathbf{C},$ are matrices describing the relationships between the three. Your application will tell you something about how to choose these matrices. I do not know mass spectroscopy at all, so I can't really give guidance here.

The Kalman filter equations can then be applied and estimates of $\underline{x}_k$ obtained.

Note that the usual application of these equations is not, technically, smoothing.

There are three applications of the Kalman filter;

• Filtering: obtaining an estimate of the state at time $k$ using all measurements of $\underline{z}_k$ up until time $k$.
• Smoothing: obtaining an estimate of the state at time $k$ using all measurements of $\underline{z}_k$ up until time $k+K$ (using $K$ future measurements).
• Prediction: obtaining an estimate of the state at time $k$ using all measurements of $\underline{z}_k$ up until time $k-K$ (looking into the future $K$ samples).

Under some (many?) assumptions, the Kalman filter is optimal:

• ♦ thanks peter but what I am very interested to know is how to understand if the smoothing or filtering technique got raid of the noise and how well it performed ? my mean which criteria should we apply to understand the performance of a given method for smoothing or filtering ? – nik May 22 '16 at 20:00
• The Kalman Filter is optimal (see my update) under various assumptions. The link goes through things in more detail. If you're concerned, I'd try the straight Kalman filter first, and then look at what a Kalman smoother buys you (or doesn't). – Peter K. May 23 '16 at 13:25