My team is working on a program that can read nutrition facts labels using OCR, much like NutriScanner (though by its reviews it doesn't seem like it works particularly well).

I've seen other questions noting that it'd be a good idea to straighten out the image prior to running OCR on it. Is there a good image library that can automatically do that? I've looked around and found numerous Photoshop tutorials on how to do it manually, but that's not what I'm looking for.

I know OCR engines like ABBYY have some pre-processing features built-in, but I'd prefer to piece together a solution using Tesseract and a free library for the pre-processing.

I haven't found much in the way of leveraging the positioning of the items on the label to improve scanning accuracy, but any suggestions would be appreciated.

  • $\begingroup$ Somehow i think the app you linked uses the same approach: tesseract and some hand-made preprocessing. In the mean time, what exactly are you asking for? $\endgroup$
    – Nikolay
    Commented May 18, 2012 at 14:54
  • $\begingroup$ They mention that trying to scan curved items (e.g. cans) doesn't work well, due to curvature warping the characters. I want to know if there's an automated way to straighten out the image to improve the OCR accuracy. Bonus points for other suggestions on customized OCR for nutrition labels, though... haven't found anything. $\endgroup$
    – ikim
    Commented May 18, 2012 at 19:56

1 Answer 1


Been there, done that. One of our on-going projects is to create a process for processing nutrition labels from iPhone camera for a health-tracking app. My company decided to develop a solution for this particular application - extracting data from US nutrition labels. This solution will be used for this client, but we decided to pre-package a few other flexible capabilities along the way, to be used by a wider audience. The solution will be available to general public in about 3 weeks. (I'll come back here and post an update.)

Main goal that will make or break the entire idea is to get usable images. Curved images, heavy shading, low resolution, and blur will all substantially decrease OCR quality, often to no quality. In apps, we found that user training and guidance is the most helpful method. Then there are some technical tools such as quickly detecting quality of image and suggesting to re-take it if needed. In general, if you can achieve "high quality fuel" (images) you can expect high performance from your "machine".

Next, the OCR. Tesseract can do good job reading clear and simple test such as pages from books. Commercial products such as ABBYY distinguish themselves by working well on tough images - shading, distortions, small prints, etc. And unfortunately if mobile images are used, they are more often on bad side than good quality side.

Next, pick your approach to locating and extracting data. Please see my answer on text parsing vs specialized test extraction tools: https://stackoverflow.com/questions/3070732/processing-ocred-text For our project we'll be using ABBYY FlexiCapture for targeted nutrition data. It has special tools even when OCR makes mistakes to still find appropriate data (sort of controlled fuzzy search), and that was an important factor in selecting it for the task.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.