1
$\begingroup$

I have some video frames I'd like to extract some text from (for making softsubs from hardsubbed video for learning). The text in question is mostly in Chinese script (some numbers and Latin letters) of a known font (looks like SimHei), is white (ish) and has a black (ish) border, and occurs left-aligned at a constant location. What I'd like to achieve is a "tidy" black-on-white image of the text.

Example input, something like this, where the background is uncontrolled, and can also be as white as the text.

enter image description here

What I'd like to achieve is something like this that can be fed to OCR:

Desired output

I am using Python and OpenCV. A naive first attempt that just thresholded the image is not sufficient, as it (predictably) produces artifacts in areas of white background (eg under the first and second characters in this example).

Edge detection is also challenging, as you get a decent "inner edge" for the white-black edge, but you might or might not get an outer edge for the black-background boundary (if the background is dark), or you might only get a partial or unclosed outline. Some backgrounds are very noisy, so there can be a lot of spurious edges too (this is a bit contrived, as no filtering is done, and the Canny aperture is too big, but the double edges are visible):

enter image description here

I have tried to filter the image with a Laplace Pyramid filter (based on this example), but it's hard to find a filter that homes on the white text without often blowing noise in the background into pure white too. E.g. on the right here is a feature that looks quite "text-y" after thresholding.

enter image description here

Filtering "small" contours is also challenging, as there are valid small strokes in Chinese, as well as punctuation like full stops and commas.

What is the right (or a good) way to robustly filter for text like this? Is there a "standard" technique that can latch onto this kind of outline? I'm hoping to stick to image processing and basic heuristics, rather than more advanced discrimination like machine learning.


Here are some more frames, not all of which have text, showing that there are some features in the image that are not very well suited to one-size-fits-all processing:

enter image description here

$\endgroup$
  • $\begingroup$ What you can get if you threshold the image and then do edge detecting? $\endgroup$ – lennon310 Jun 27 at 17:52
  • $\begingroup$ It works "OK", but it's still quite vulnerable to adverse backgrounds: imgur.com/a/ogVimbs is an example where the background is creeping in, even as the text is being damaged. $\endgroup$ – Inductiveload Jun 27 at 18:01
1
$\begingroup$

You can obtain pretty good results by just thresholding the image at a high intensity (since your text appears always to be white) and do a closing operation to close the gaps:

# convert to grayscale
img = cv2.imread('OCR.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshhold
ret,bin = cv2.threshold(gray,245,255,cv2.THRESH_BINARY)

# closing
kernel = np.ones((3,3),np.uint8)
closing = cv2.morphologyEx(bin, cv2.MORPH_CLOSE, kernel)

# invert black/white
inv = cv2.bitwise_not(closing)

Result:

enter image description here

You can probably clean up most background artifacts with an opening operation of the inverted image. This should already yield pretty good OCR results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.