1
$\begingroup$

I have some data $(X,Y,Z)$, which is a set of measurements $Z$ over a $2D$ space $X$,$Y$. The $Z$ data on this space is continuous except for some discontinuous jumps in certain domains of $X$,$Y$ (due to calibration errors).

For example, let's say $X$ and $Y$ are both between $[0,1)$, and let's say my underlying true function is $Z=f(X,Y)$. For most of the $X$,$Y$ space $Z$ will equal my true function, but perhaps for $0.25 < X < 0.5$, $0.25 < Y < 0.5$, my data will show $Z=f(X,Y)+4$, an offset of $4$ from the true function.

I can brute-force look for discontinuous jumps and then correct for them in the data set (by setting the edge points in the error region equal to their nearest points in the non-error region), but this can be problematic for certain functions and certain domains that I'm working with. I'm wondering if there's a signal processing method that is designed to work for these types of problems?

Edit: To be clear, I only need to learn the function $f(X,Y)$ up to a constant, so it doesn't matter what part of the space I consider the "error-free" region or not. I simply can't have the function exhibiting different offsets in different regions of the 2D $X,Y$ space.

$\endgroup$
  • $\begingroup$ Do you mean change in the DC level? $\endgroup$ – Royi May 12 '15 at 5:31
  • $\begingroup$ Sure, that would probably be an equivalent way of describing it. $\endgroup$ – gammapoint May 12 '15 at 13:11
  • $\begingroup$ OK, I have an idea... $\endgroup$ – Royi May 12 '15 at 13:47
1
$\begingroup$

Try a Median Filter with a large enough sliding window, to pull the outliers (discontinuous jumps) back within tolerance.

$\endgroup$
  • $\begingroup$ Thanks for the suggestion. I'm skeptical if this will work since the outlier regions can be of the same size as the "good" regions, but it will be interesting to give it a try and see how it looks. For some of the outlier regions that are smaller this may be a good solution. $\endgroup$ – gammapoint Oct 14 '14 at 21:30
  • 1
    $\begingroup$ Do you have an example signal to show? I agree that if your outlier regions are the same size as the "good" reasons, the dsp techniques that I might suggest, which typically fall into the de-noising, smoothing category will not apply.You may need to try a statistical or machine learning technique. $\endgroup$ – ruoho ruotsi Oct 17 '14 at 18:04
1
$\begingroup$

If I understand you correctly you want to smooth the data (Namely reduce "Noise") yet regular filters would ruin the data on discontinuities.

What you need is an Edge Preserving Filter.
You can try the Bilateral Filter or Anisotropic Filter.
I have an advanced implementation of the Anisotropic Filter - Fast Anisotropic Smoothing of Multi Valued Images Using Curvature Preserving PDE.

I hope it works.

$\endgroup$
  • $\begingroup$ Thanks for the suggestion. I'm currently not working on this particular problem anymore (its business-relevance has declined since then), but if I need to revisit it I look forward to trying out your edge-preserving filter. $\endgroup$ – gammapoint May 19 '15 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.