5
$\begingroup$

I have some data $(X,Y,Z)$, which is a set of measurements $Z$ over a $2D$ space $X$,$Y$. The $Z$ data on this space is continuous except for some discontinuous jumps in certain domains of $X$,$Y$ (due to calibration errors).

For example, let's say $X$ and $Y$ are both between $[0,1)$, and let's say my underlying true function is $Z=f(X,Y)$. For most of the $X$,$Y$ space $Z$ will equal my true function, but perhaps for $0.25 < X < 0.5$, $0.25 < Y < 0.5$, my data will show $Z=f(X,Y)+4$, an offset of $4$ from the true function.

I can brute-force look for discontinuous jumps and then correct for them in the data set (by setting the edge points in the error region equal to their nearest points in the non-error region), but this can be problematic for certain functions and certain domains that I'm working with. I'm wondering if there's a signal processing method that is designed to work for these types of problems?

Edit: To be clear, I only need to learn the function $f(X,Y)$ up to a constant, so it doesn't matter what part of the space I consider the "error-free" region or not. I simply can't have the function exhibiting different offsets in different regions of the 2D $X,Y$ space.

$\endgroup$
3
  • $\begingroup$ Do you mean change in the DC level? $\endgroup$
    – Royi
    May 12, 2015 at 5:31
  • $\begingroup$ Sure, that would probably be an equivalent way of describing it. $\endgroup$
    – gammapoint
    May 12, 2015 at 13:11
  • $\begingroup$ OK, I have an idea... $\endgroup$
    – Royi
    May 12, 2015 at 13:47

2 Answers 2

1
$\begingroup$

If I understand you correctly you want to smooth the data (Namely reduce "Noise") yet regular filters would ruin the data on discontinuities.

What you need is an Edge Preserving Filter.
You can try the Bilateral Filter or Anisotropic Filter.
I have an advanced implementation of the Anisotropic Filter - Fast Anisotropic Smoothing of Multi Valued Images Using Curvature Preserving PDE.

I hope it works.

$\endgroup$
1
  • $\begingroup$ Thanks for the suggestion. I'm currently not working on this particular problem anymore (its business-relevance has declined since then), but if I need to revisit it I look forward to trying out your edge-preserving filter. $\endgroup$
    – gammapoint
    May 19, 2015 at 23:06
2
$\begingroup$

Try a Median Filter with a large enough sliding window, to pull the outliers (discontinuous jumps) back within tolerance.

$\endgroup$
2
  • $\begingroup$ Thanks for the suggestion. I'm skeptical if this will work since the outlier regions can be of the same size as the "good" regions, but it will be interesting to give it a try and see how it looks. For some of the outlier regions that are smaller this may be a good solution. $\endgroup$
    – gammapoint
    Oct 14, 2014 at 21:30
  • 1
    $\begingroup$ Do you have an example signal to show? I agree that if your outlier regions are the same size as the "good" reasons, the dsp techniques that I might suggest, which typically fall into the de-noising, smoothing category will not apply.You may need to try a statistical or machine learning technique. $\endgroup$ Oct 17, 2014 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.