# Generic text recognition

OCR is a quite an old subject in Image Processing & Vision. However, the most literature that I had come across 1, 2, 3 shows various matured techniques which I am sure will give some good results in document processing.

Unfortunately, most OCR begins with document and that's when its utility gets limited. If you look at more generic scenario when a person goes to airport figures out where the sign board is and "reads" the signs board; now if we want to get a robot to achieve something like this, how does one go about achieving this?

Is there a more generic "text recognition" systems and algorithms?

• Why downvote? If you disagree learn to put your words... Jun 21 '17 at 18:57

A short answer to your question is "NO". To my best knowledge, there is no such a smart end-to-end OCR system that can handle various OCR tasks. However, there are OCR products could reasonably handle some tasks under controlled environments.

You seem to be confused of different OCR related tasks and you erroneously believe these tasks are quite similar ( and thus something works well on document could also work well on reading signs ).

Here lists several levels of OCR tasks:

• level 0: single character recognition
• level 1: word recognition level
• level 2: sentence recognition
• level 3: paragraph recognition
• level 4: document recognition
• level 5: OCR in the wild

Depending on the source of characters, you may classify texts into 1) machine printed texts ; 2) handwritten texts; and 3) scene text. All these are OCR related tasks, and thus can all be called "OCR".

These tasks are clearly of different difficulties. Basically, the higher the level the more difficult it is. More importantly, all these tasks are NOT just about recognition. Sometimes pre- and post- processing are both important. For example, it is well known that the line segmentation problem in handwritten document recognition is very challenging when text lines are cursive, or touched, or mixed of texts from different directions, etc. If you cannot first segment text line correctly, how to recognize them? For another example, it is also well known that deciding the reading order of multi-column document mixed with pictures is challenging. Even if you recognize every sentence correctly, you may still make mistakes in organizing different paragraphs. In your interested scene text recognition tasks, text localization itself is challenging in the sense that texts can be any font in anywhere with any direction and perspective change.

For these reasons, a good end-to-end OCR engine is at least a mix of image processing, pattern recognition, and natural language processing, and thus this is a quite big system requiring knowledge from all these parts.

I put my comments of 'prior knowledge' below. To discuss 'prior knowledge', it is better to clarify its definition. - For recognition, it often means there is prior distribution to tell you how likely a character is expected to see. If you said you have no prior knowledge, then you believe it is equally likely to see all possible characters. Think about how many valid characters in all languages. This number is big, and the number of valid words in all languages is even bigger. - For image processing, it often means you have (explicit or implicit) assumptions about a given image. Such assumptions could be 'one-column document', 'car plate', 'taken by iphone', etc. If you say I have no prior knowledge about where a car plate is, then this simply means you treat the possibility of seeing a car plate is equally likely for any image patch size and location. Think about how many possible image patches you will have.

It is clear that doing OCR without prior knowledge is the worst thing we can try. In other words, we could do better when we have more prior knowledge for our task. In practice, people are using various prior knowledge, but often in an implicit way, and thus they even donot notice it.

One typical example is training data. Because your training data is not the entire universe, any rules, distributions, characteristics etc you learnt from this dataset may not necessarily apply to the rest of the world. However, you are still use your trained models to do various tasks. Your prediction is based on the prior knowledge that you learnt from data.

Back to your 'car plate' OCR problem. We know a car plate is of a rectangular shape under some perspective transformation. We know it is very likely to locate at the lower part of a car. We know it cannot be to too thin nor too small. Such prior knowledge helps quickly locate a candidate plate while avoiding a lot false alarms. When you recognize a plate, there is more noticeable prior knowledge. If you know only digits are allowed in your plate, then you cut your solution space to the digit space. Even if you see something more similar to letter 'g' than digit '9', you can safely say "this must be '9'" because of your prior knowledge. For the same reason, if you know plate are valid, you can safely thrown away any candidate solution that is invalid. Even 'not that useful prior knowledge' helps. For example, if you know no plate in our world has more than N characters, you know a plate candidate you are recognizing is NOT a valid plate because its length is above N.

If you donot have any prior knowledge, what happens? For me, it is similar to ask me find some object named '%#' on an image, which I completely have no idea what it is. And this is an impossible task. However, if I have the knowledge that '%#\$#' is the ciphertext of 'ball', then I can easily do this task.

I hope that I convince you that prior knowledge is useful and closely associated with each processing step, and we cannot do OCR without prior knowledge. Indeed, this is the exact reason why we need to combine models from different aspects for OCR.

Donot be fooled by the name of OCR "optical character recognition". It is not just character recognition: we are actually reading words, sentences, paragraphs from an image that is taken by someone using some particular device... and thus we need prior knowledge at many different levels.

• I apprecieate to put the whole thing in better perspective. And also, I am aware of algorithms that apply to machine vision or say number plate reading etc. which essentially do land up. I think the crux of my question is try to do OCR where the scene has no prior knowledge. How does one do that? Also, NO is really not a good answer - because I am sure I am not the first one who would have thought of it! So may be will search more... Jun 15 '15 at 14:52