# smooth noisy irregularly spaced data containing peaks

I've got a set of scans of an object (human body) from different angles, which are being combined to reconstruct a 2D-representation. The raw measurements (blue plot below) contain a fair amount of spikes, which I reduce by using a median filter (result in green plot below). Afterwards I filter the result with a FIR lowpass filter to further remove noise (red figure below). This produces at least visually appealing results, though I don't know if it is the best possible solution. Are there any pitfalls, when chaining median and linear filtering?

Another problem is, the samples are not evenly spaced. The distribution of samples seems to be somewhat $1/x$-ish, though I don't take this into account when smoothing the data. Can this be ignored for simple smoothing? If yes, why? If no, how can I factor in the irregular spacing?

• Your present approach sounds sound. Just to be sure: is your data a series of couples $(\phi,r)$? Do you median/FIR filter $r$ like it were a 1D signal, regularly spaced? – Laurent Duval Feb 2 '17 at 9:41
• Yes, my data consists of $(\varphi, r)$-tuples, and I filter them as if they were regularly spaced 1D-data – Finwood Feb 2 '17 at 9:48
• Do you have a metric to assess the performance of your filtering? What are you using your clean for afterward? Shape factor? Comparisons? Could the angle be affected by noise too? – Laurent Duval Feb 2 '17 at 9:55
• So far, my metric is "looks good", nothing substantial. The smoothed data will then be used to determine (for now) the circumference of the scanned body, and the angle will most likely also be affected by noise. – Finwood Feb 2 '17 at 10:00

The idea of cascading a non-linear then a linear filter is sound in the presence of spikes.

Here are some observations which, if correct, could be exploited for better processing:

• Data is circularly arranged, and in fact 2D
• Spikes point outward,
• The remaining noise seems bounded (the thick blue ring),
• The angular sampling is dense, and could be affected by noise.

Some suggestions:

• Choose a novel angular sampling $\delta \phi$ that could be used for several datasets. It could be parametrized by a modelling of histograms with a one-sided exponential law, like $a\exp(-\lambda t)$, $t\ge 0$.
• Choose an angular width $\Delta \phi$, that will be used as the span of a 2D filter.

Their combination would define a series of overlapping windows on which you would process 2D point clouds.

Inside each angular region defined by $[n\delta \phi - \Delta \phi/2, n\delta \phi + \Delta \phi/2]$, you can for instance:

Introducing a weighting in a median filter can bridge the gap between pure median and smoothing filters, and kill two birds with one stone: remove spike, reduce noise. As a result, this could suffice. If not, you can then:

• perform a 2D linear filtering on the remaining points, in the shape of a center of mass allocation.

One advantage with resampling to a regular angular grid is potentially to allow easiest further processing, comparisons, etc. And using 2D filters, you naturally follow the circular symmetry of your data.