1
$\begingroup$

I currently have an invoice, where some numbers are written by hand, and I would need to read them automatically.

My past experience is that people who write numbers will them write in so many different ways (overlapping with field name etc.) that it's really hard to isolate only digits.

Now I would like to redesign invoice, for example, to add field of some rectangular shape, that will be easily detectable (think easy to detect like a QR code is).

I'm not sure:

should I add some ordinary box, or are there any specific designs of fields/boxes/rectangles that would make finding the field and isolating everything in it (from image) simple and 99% correct?

Tech stack that would be used is:

  • OpenCV
  • Python
  • HTML/CSS for creating the design of invoice, that would be exported to pdf for users to fill it with the needed information
Improve this question
$\endgroup$
1
  • $\begingroup$ If you are trying to recognise the characters themselves, then this question is not about "rectangular fields" but about Optical Character Recognition directly. Can you please edit your question so that it reflects exactly what you mean? $\endgroup$
    – A_A
    Commented Jan 24, 2019 at 9:13

1 Answer 1

Reset to default
1
$\begingroup$

There's been forms like that forever: In Germany, the Überweisung (wire transfer) / Verrechnungsscheck (Cheque for accounting) have always been on things like this:

Zahlschein.jpg
Link

and as far as I can tell, that works pretty well with 1990's tech.

Note how the boxes are part of the "background" and are color-wise probably designed to simply disappear in a first processing step after scanning.

However, while that might be have been at a different level of initial investment, postal services find and read postal codes / ZIP numbers that are handwritten since at least the mid-nineties, so: Yeah, bummer that people aren't good OCR-compatible writers, but maybe your OCR just has to get better: Knowing that you're only looking for numbers will help a lot. Don't take an OpenCV example as a complete OCR solution. There's more to OCR than segmenting individual glyphs and trying to find the closest match to each one from an alphabet in isolation.

Improve this answer
$\endgroup$
17
  • $\begingroup$ could you give me some hint for ` Don't take an OpenCV example ...` and There's more to OCR than segmenting... I would advise with google after your advice, but currently I know only for segmenting, binarization etc. I was thinking and looking into maybe frequency matching but didn't know where to go from there. Currently, I use things like morphology, pattern matching, convex hull/bounding boxes etc. So maybe I could really make my OCR better, the problem is I'm kind of new to the whole image processing thing so It could be that I don't see the solution. $\endgroup$
    – Igor
    Commented Jan 23, 2019 at 8:21
  • $\begingroup$ You're thinking of OCR being image processing (e.g. filtering, binarization, segmentation) and then glyph recognition (pattern matching of sorts). What it really is is these two, but with a lot of surrounding logic: For example, if you can find a number that your algorithm is 100% sure is a 0, then that shifts the likelihoods about how a 6 would look like written by the same person. Also, you would now compare a different number on the same form that you think might be a 0 with that known to be good 0 and get more confidence. You'd also not look at the digits in isolation – $\endgroup$
    – Marcus Müller
    Commented Jan 23, 2019 at 9:06
  • $\begingroup$ you'd, for example, use knowledge of the fact that nobody writes amounts of money on invoices with more than two digits after the decimal point. If you find something that looks like 104.203€ on an invoice for a car repair, then you can be pretty sure that it's likely that you misdetected the . and this is most likely 1042.03€. In "prose" OCR, you'd have dictionaries, you'd have markov models that have knowledge of which words are likely to appear in sequence, and so on. OCR is being "smarter" than just isolating digits and guessing them individually. $\endgroup$
    – Marcus Müller
    Commented Jan 23, 2019 at 9:09
  • $\begingroup$ just to give you little more context, a goal is to isolate the number, then segment those numbers and feed them to Convolutional Neural Network that would tell me what number it is. That's why I need segmentation because CNN can predict only 10 classes of digits [0,....,9] , and the problem is really getting clean numbers to feed them to CNN, as it is sensitive to noise that may come with surroundings of the number filed. $\endgroup$
    – Igor
    Commented Jan 23, 2019 at 9:22
  • $\begingroup$ so, as said: see the image of the form I used. But your CNN shouldn't have 10 classes, it should be giving you a number at the end, with multiple digits (what you want to solve is reading numbers, not digits). again: Higher order intelligence is necessary here, not just identifying isolated digits. You don't have a classifying problem (though classification might be helpful on the way to the solution), but a scalar problem: Just because CNN are most popularly used as classifiers doesn't mean that's what they must do: Many CNNs simply output numbers. Yours should probably output cash amounts $\endgroup$
    – Marcus Müller
    Commented Jan 23, 2019 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.