I want to remove the background from image. Let's say I have this background:

sample background image.

Then I put pen drive and tippex on it.

image with objects.

Is it possible to remove the background image from second image? If so, then how can I do it?

I'm using the AForge.NET Framework.


I want to remove background like in this research paper. Is there any library for this kind of work?.

  • $\begingroup$ @Phonon: actually I don't use the above background at the movement. I use plain color background and use ColorFilter to remove it. $\endgroup$
    – Darshana
    Commented Nov 4, 2013 at 1:57
  • $\begingroup$ @Darshana What about subtracting the current frame from previous frame ? or backgroungsubtractor mog2 ? $\endgroup$
    – ARG
    Commented Nov 10, 2013 at 10:28

2 Answers 2


Since you already know your background image beforehand, it should be simple. I done many background subtraction before. Here is an example of removal of background by comparison.

    if (newBitmap.Width == backgroundBitmap.Width && newBitmap.Height == backgroundBitmap.Height)
        for(int x = 0; x < newBitmap.Width; x++)
            for (int y = 0; y < newBitmap.Height; y++)
                Color firstPixel = newBitmap.GetPixel(x, y);
                Color SecondPixel = backgroundBitmap.GetPixel(x, y);  

                int Totalfirst = (int)((firstPixel.R * .3) + (firstPixel.G * .59) + (firstPixel.B * .11));
                int Totalsecond = (int)((SecondPixel.R * .3) + (SecondPixel.G * .59) + (SecondPixel.B * .11));
                if (Totalfirst < Totalsecond )

                    Color WhiteColor = Color.FromArgb(0, 0, 0);
                    newBitmap.SetPixel(x, y, WhiteColor);
                    Color SameColor = Color.FromArgb(firstPixel.R, firstPixel.G, firstPixel.B);
                    newBitmap.SetPixel(x, y, SameColor);

    FinalImage = newBitmap; // after substaction of back ground.

This code is just a copy and paste from one of the program I did back then. Note that newBitmap is the image of the object and the background, your second picture. While the backgroundBitmap is well, the background.

Use this as your reference on how your code should look like for this segment. Should be good enough.

Also, I am new to this website(my first post here), but I think for image processing questions, you can post it to https://stackoverflow.com/questions next time, there will be more people to help you.

If this method doesn't work, let me know, cause I still got some other methods. But from you question, this should be sufficient.

  • $\begingroup$ what are these .3, .59 and .11? $\endgroup$
    – Darshana
    Commented Nov 12, 2013 at 6:12
  • $\begingroup$ please explain your algorithm first. $\endgroup$ Commented Nov 12, 2013 at 18:30
  • $\begingroup$ the .3, .59 and .11 are the RGB luminance value. You can find out more from these links: scantips.com/lumin.html and groups.google.com/forum/#!topic/xsi_list/bv3y0WX2R9Q. While this is supposedly the default, there are many people who uses their own values too. Hope that helps(: $\endgroup$ Commented Nov 12, 2013 at 18:40
  • $\begingroup$ @AbidRahmanK, My algorithm flow and idea is pretty simple and straightforward. Take background image(1) and background image with foreground objects(2). Compare pixel by pixel. If the pixel are the same, make it to black, eliminating it. Whatever remains is the odd ones out, basically the foreground objects. Hope that helps (: It's a very common method. $\endgroup$ Commented Nov 12, 2013 at 18:45
  • $\begingroup$ @rockinfresh: What I wanted to say is, while answering, first explain the algorithm, then provide its code (as given in the second answer here). Most of the time, readers won't be that experienced or even newbies in the field of image processing. So the explanation will help them. $\endgroup$ Commented Nov 12, 2013 at 18:51

From an algorithm perspective, subtraction is the simplest method.

  1. Run a feature detection algorithm (like SIFT or SURF) on both images to align them
  2. Apply any enhancement operations at this stage, such as lens distortion removal, lighting correction, or others.
  3. Do a simple subtraction of the background image from the test image.
  4. Perform a blur or median filter on the subtracted image. This will mitigate noise from the subtraction.
  5. Run a threshold tool on this image to generate a foreground mask.
  6. Optionally blur the foreground mask to smooth out the edges.
  7. Apply the mask to your test image and only the foreground will be visible.

The following is some ugly code that accomplished this crudely. If I have time I will clean up the code, or perhaps someone else can for me. I used OpenCV, which is very capable for doing what you want, though you would need to use the .NET bindings.

import cv
import cv2
import numpy as np
import sys

# Align images with cv2
imgBack = cv2.imread("Background.png")
imgTest = cv2.imread("Test.png")

detector = cv2.FeatureDetector_create("SURF")
descriptor = cv2.DescriptorExtractor_create("BRIEF")
matcher = cv2.DescriptorMatcher_create("BruteForce-Hamming")

# the matcher works on single channel images
grayBack = cv2.cvtColor(imgBack, cv2.COLOR_RGB2GRAY)
grayTest = cv2.cvtColor(imgTest, cv2.COLOR_RGB2GRAY)

# Compute magic (I found this code elsewhere 
kp1 = detector.detect(grayBack)
kp2 = detector.detect(grayTest)

k1, d1 = descriptor.compute(grayBack, kp1)
k2, d2 = descriptor.compute(grayTest, kp2)

matches = matcher.match(d1, d2)

dist = [m.distance for m in matches]
if len(dist) == 0:
    print "No Suitable matches found!"

mean_dist = sum(dist) / len(dist)
threshold_dist = mean_dist * 0.5

good_matches = [m for m in matches if m.distance < threshold_dist]

h1, w1 = grayBack.shape[:2]

matches = []
for m in good_matches:
    matches.append((m.queryIdx, m.trainIdx))

if len(good_matches) > 0:
    p1 = np.float32( [k1[i].pt for i, j in matches] )
    p0 = np.float32( [k2[j].pt for i, j in matches] )
    H, mask = cv2.findHomography(p0, p1, cv2.RANSAC)

    output = cv2.warpPerspective(imgTest, H, (w1, h1))
# End Magic

cv2.imwrite("Test-Warped.png", output)

imgBack = cv.LoadImage("Background.png")
imgTest = cv.LoadImage("Test-Warped.png")

diff = cv.CloneImage(imgBack)
grayDiff = cv.CreateImage(cv.GetSize(imgBack), cv.IPL_DEPTH_8U, 1)

# Perform simple substraction    
cv.AbsDiff(imgBack, imgTest, diff)
cv.CvtColor(diff, grayDiff, cv.CV_RGB2GRAY)

# Get our threshold and expand it a little to clean the edges
cv.Threshold(grayDiff, grayDiff, 40, 255, cv.CV_THRESH_BINARY)
cv.Dilate(grayDiff, grayDiff, None, 6)
cv.Erode(grayDiff, grayDiff, None, 2)

# Copy the test image and mask with our threshold
fore = cv.CreateImage(cv.GetSize(imgBack), cv.IPL_DEPTH_8U, 3)
cv.Copy(imgTest, fore, grayDiff)

cv.SaveImage("Threshold.png", grayDiff)
cv.SaveImage("Foreground.png", fore)

Intermediate threshold found by algorithm Final (crude) extracted foreground

  • $\begingroup$ good explanation. but I'm not that much familiar with C++ and OpenCV. thanx. $\endgroup$
    – Darshana
    Commented Nov 14, 2013 at 5:49
  • $\begingroup$ @Darshana - not a problem! You can use one of several .NET bindings for OpenCV to use .NET and OpenCV has a pretty good community showing tutorials and great documentation. $\endgroup$
    – dmsnell
    Commented Nov 15, 2013 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.