Take the 2-minute tour ×
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It's 100% free, no registration required.

I am trying, for my own learning purposes, to develop an implementation of an algorithm that would list books, given an image of a bookshelf like the following:

enter image description here

The first step is to slice the image into individual books.

My algorithm, in Mathematica, is:

    img = ColorConvert[Import["http://i.stack.imgur.com/IaLQk.jpg"], "GrayScale"]
  • do a basic edge detection &
  • remove the text and try to keep the long lines

    edge = DeleteSmallComponents[EdgeDetect[img, 3],Last[ImageDimensions[img]]/5]
    
  • then remove unwanted horizontal lines

    lines = Sort[ImageLines[img5] /. {{0., _}, {_, _}} -> Sequence[]]
    Show[img, Graphics[{Thick, Orange, Line /@ lines}]]
    

The results, though, are less than great:

enter image description here

My questions are:

  1. How can I improve this to yield better results?
  2. Is there a smarter way to do this?
  3. how further should I process the images to increase accuracy at the (later) OCR phase?
  4. How to use the color information to improve the segmentation?
share|improve this question

migrated from stackoverflow.com Nov 15 '12 at 14:13

This question came from our site for professional and enthusiast programmers.

1  
@OrenPinsky I don't see the problem with the segmentation: the number of "false book areas" is not high (I can see only one in the sample you provided), and if you plan to do some kind of text recognition as the next step, that should be enough to discriminate between books (area has text) and not-books (no text) –  penelope Nov 15 '12 at 16:02
3  
@OrenPinsky, I would say your results are pretty great. :) –  Dima Nov 15 '12 at 22:15

3 Answers 3

Here is the link to a research paper that tries to do the same thing as you wanted. It might help you.using image features Also a cool video on the youtube

share|improve this answer

Which method are you using to detect the lines? Have you tried experimenting with LSD?

Here are the results of a quick test I did using LSD:

In this first image I have displayed only the vertical line segments with an angle between 75 and 105 degrees and the length greater than $0.1 * height$ of the image: pic1

The second image are the results with the same angle constraint but disregarding the lengths of the segments: pic2

You can try playing with this a bit, figure out how to chose the best line segments, extend them to lines and maybe get slightly better results than the ones you posted.

share|improve this answer
3  
"Have you tried experimenting with LSD?" Nice try, FBI ;) –  Lorem Ipsum Nov 18 '12 at 20:51
    
Mathematica's ImageLine is based on the Hough transform, and I am now convinced (from the feedback here, mostly) that it works pretty well. It bothers me, though, that I am loosing relevant data when I transform into grayscale, and that in this application color data could be (intuitively) help a robust edge detector.Will try LSD and see how it goes! (it worked amazingly well for Steve Jobs! ;-) –  Oren Pinsky Nov 19 '12 at 2:16
    
I've seen a friend use LSD for door detection, I think he was pleased with the results in the end. I'd say it's worth a shot :D –  penelope Nov 19 '12 at 9:55

You can try doing edge detection on individual color domains and then merge them, using your method of choice for edge detection.

Compared to edge detection directly on the color image, it might produce better results.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.