# Slicing and Recognition of book title and author from bookshelf image

I am trying, for my own learning purposes, to develop an implementation of an algorithm that would list books, given an image of a bookshelf like the following:

The first step is to slice the image into individual books.

My algorithm, in Mathematica, is:

    img = ColorConvert[Import["http://i.stack.imgur.com/IaLQk.jpg"], "GrayScale"]

• do a basic edge detection &
• remove the text and try to keep the long lines

edge = DeleteSmallComponents[EdgeDetect[img, 3],Last[ImageDimensions[img]]/5]

• then remove unwanted horizontal lines

lines = Sort[ImageLines[img5] /. {{0., _}, {_, _}} -> Sequence[]]
Show[img, Graphics[{Thick, Orange, Line /@ lines}]]


The results, though, are less than great:

My questions are:

1. How can I improve this to yield better results?
2. Is there a smarter way to do this?
3. how further should I process the images to increase accuracy at the (later) OCR phase?
4. How to use the color information to improve the segmentation?
-

## migrated from stackoverflow.comNov 15 '12 at 14:13

This question came from our site for professional and enthusiast programmers.

@OrenPinsky I don't see the problem with the segmentation: the number of "false book areas" is not high (I can see only one in the sample you provided), and if you plan to do some kind of text recognition as the next step, that should be enough to discriminate between books (area has text) and not-books (no text) –  penelope Nov 15 '12 at 16:02
@OrenPinsky, I would say your results are pretty great. :) –  Dima Nov 15 '12 at 22:15

Here is the link to a research paper that tries to do the same thing as you wanted. It might help you.using image features Also a cool video on the youtube

-

Which method are you using to detect the lines? Have you tried experimenting with LSD?

Here are the results of a quick test I did using LSD:

In this first image I have displayed only the vertical line segments with an angle between 75 and 105 degrees and the length greater than $0.1 * height$ of the image:

The second image are the results with the same angle constraint but disregarding the lengths of the segments:

You can try playing with this a bit, figure out how to chose the best line segments, extend them to lines and maybe get slightly better results than the ones you posted.

-
"Have you tried experimenting with LSD?" Nice try, FBI ;) –  Lorem Ipsum Nov 18 '12 at 20:51
Mathematica's ImageLine is based on the Hough transform, and I am now convinced (from the feedback here, mostly) that it works pretty well. It bothers me, though, that I am loosing relevant data when I transform into grayscale, and that in this application color data could be (intuitively) help a robust edge detector.Will try LSD and see how it goes! (it worked amazingly well for Steve Jobs! ;-) –  Oren Pinsky Nov 19 '12 at 2:16
I've seen a friend use LSD for door detection, I think he was pleased with the results in the end. I'd say it's worth a shot :D –  penelope Nov 19 '12 at 9:55

You can try doing edge detection on individual color domains and then merge them, using your method of choice for edge detection.

Compared to edge detection directly on the color image, it might produce better results.

-