0
$\begingroup$

I'm really new in image processing, so please sorry I'm a newbie. I tried to use the squares.cpp for detecting posters (since they usually are rectangles) without using expensive feature detectors (like SIFT). Unfortunately, the results are pretty much disappointing (as it was pretty predictable, results below).

Actually I don't care that only posters are detected, since statistically the posters are the biggest (or second biggest) rectangle in the image (decent heuristic).

The last image is the result of this code using this Hough Transofrm code (which seems working even worse!).

Any idea how to improve this code?

enter image description here

[enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ Well, there's something wrong with the Hough detector; there's no horizontal lines in the picture where you've got horizontal lines of dots; you will need to investigate that if you want to actually try Hough detector (not only look at a single piece of code and decide that it doesn't work). $\endgroup$
    – Marcus Müller
    Commented May 28, 2016 at 8:33
  • $\begingroup$ Have a look at the output just after Canny edge detection and see if there are too many lines. $\endgroup$
    – geometrikal
    Commented May 30, 2016 at 3:33

1 Answer 1

Reset to default
1
$\begingroup$

It seemed to work relatively well where there was a strong lightness difference at the edge of the poster, compared to background.

This matches what squares.cpp seems to do: scale the image, do a canny edge detection, simplify the resulting contours and look for squares.

There's two things that, looking at your example images, go wrong here:

  • The contour detection / simplification doesn't seem to deal well with reflections crossing the edge (see Cpt America poster in last photo, and the batman vs superman poster in the 4th image)
  • The canny edge detector will fail (and so does my brain) on posters when the frame/background difference isn't clear. I had to look twice myself at the second picture to make sure where the chrome frame ends and where the poster begins

So, this points to it being helpful to pre-process your images in different way, try to detect the rectangle, change your preprocessing, repeat, and so on.

A few observations:

  • maybe you can just reduce the aggressiveness of the contour simplification
  • Try different edge detectors! Canny is not always the best choice.
  • the weaker reflections just seem to be relatively white light – maybe convert your image to a brightness/hue/saturation colorspace, and reduce the variance of the brightness channel

As a side note: this approach (pyramid scaling, edge detection, contour extraction, contour simplification, rectangle detection) sounds more complex than just a normal Feature extraction by means of a transform of the image. For example, there are several transforms in OpenCV, some of which are very usable to detect dominant straight lines in the picture. If you can find parallelograms in those, you should be one step ahead, especially if you can really assume the poster is the biggest rectangle.

I think the Hough transform is quite exactly what you need.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for your detailed answer! I already tried this code which uses the Hough transofrmation and it seems that works even worse (I'll update the question adding the results using it)! $\endgroup$
    – user6321
    Commented May 28, 2016 at 7:46
  • $\begingroup$ @user6321 well, every piece of code you'll find on the internet needs a bit of tweaking to fit exactly your needs; I wouldn't expect an answer from here to solve any problem, but instead it should give you a direction to investigate! $\endgroup$
    – Marcus Müller
    Commented May 28, 2016 at 7:48
  • $\begingroup$ I think that unfortunately you're right. Anyway I updated the question adding as last image the result of the linked code $\endgroup$
    – user6321
    Commented May 28, 2016 at 7:52
  • $\begingroup$ What about using a set of training images as done for this road sign detector using dlib where the set of training images is the poster that we are looking for (maybe in different sizes, proportions and resolutions?) $\endgroup$
    – user6321
    Commented May 28, 2016 at 7:55
  • $\begingroup$ That does sound a feasible thing in the long term, but road signs are much easier to detect, because you can first increase the contrast on an image very much and then turn it to black&white, and you'd still recognize the sign; to find more complex things in pictures: scale-invariant feature detection. $\endgroup$
    – Marcus Müller
    Commented May 28, 2016 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.