I have 2D measurement results which seem to have a constant non-trivial background as shown here:

Example for the 2D data set

For every column in this image, there seems to be a similar oscillation. I'd like to distinguish this oscillation from the real measurement signal, e.g. the oblique band which goes from about (15,1080) to (25,1140) in the image coordinates.

I already tried several things, including: FFT filtering for each individual column; using a column where no additional signal seems to be for normalization; and independent component analysis (ICA) implemented in python's scikit learn. With ICA, I was able to find a quite satisfying result, as demonstrated in this graph:

Result using ICA

where each row of the above image is plotted as a red line and the ICA result as a blue line. Unfortunately, the ICA result needs to be rescaled and it gives differing reults from time to time.

Could anyone give some hints on how to solve a task like this and how to substract the background properly and automatically in the end? Python code is highly appreciated.


  • $\begingroup$ Is this real-valued data? If you feel good about the overall shape of your background estimate (the blue line in your plot), one option would be to try calculating the vector rejection of the background from each row. That is, calculate the projection of each row vector onto the background vector, then subtract that component from the row. $\endgroup$ – Jason R May 9 '16 at 13:46
  • $\begingroup$ This sounds quite good. Yes, the data is indeed real-valued. I'll give it a try, but it still doesn't solve the problem with the ICA giving me sometimes a result a, and sometime 1/a. I'd like to have an automatic approach. Thank you, anyway! $\endgroup$ – physicsPyUser May 9 '16 at 13:51
  • $\begingroup$ If your data is zero-mean, then one way to estimate the background would be to do a framewise average of your entire dataset (i.e. average down the columns of your raster image). $\endgroup$ – Jason R May 9 '16 at 13:53
  • $\begingroup$ A fully automatic background removal, what a dream! Can you share the data somehow? $\endgroup$ – Laurent Duval May 9 '16 at 17:47
  • $\begingroup$ I provided the data on a bitbucket repository. There is only one pickle file which you can import using python pandas (pandas.read_pickle('example_data.pkl')). First experiments using the vector rejection look really promising. Unfortunately, I cannot provide another image due to a lack in reputation :/ $\endgroup$ – physicsPyUser May 10 '16 at 6:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.