I have video with text which can be rotated in any direction. If I trained a neural network on images of each letter at all possible degrees of rotation, would it be possible to classify the characters accurately? Alternatively would another technique like HAAR, or KNN be better?
Aha, this is a quite funny story that I've heard about CV. Are you a bio guy? Any way, here are my suggestions.
- If you are a bio guy and just want to finish this project ( I mean successfully identify each insect in a video frame ), go for barcode, QR etc. They are labels though their contents are not directly readable by eyes. However, you will have a much better chance to handle all kind of problems you might encountered when you use manual labels, e.g. occlusion, error correction, lighting etc. There are libraries can decode these labels, and you simply call them.
In this way, you will save a lot of energy to not working on something you are in lack of experience, but use the state-of-the-art technologies.
- If you are not a bio guy or these manual labels are not something can be changed, you should first locate these labels and then recognize them. Donot mess up these two things. The first task is detection, and the second one is recognition. In other words, you need to do two things: a). Train a text/none-text classifier. In this classifier you can use those rotational invariant features. b). Train a character classifier ( e.g. using NN ) to decide the class of a character candidate you found from (a).
By the way, I donot believe you need to train a NN accepting character at an arbitrary angle. Instead, all you need is to train a NN accepting normal characters, and normalize a candidate using image moments or other techniques before you feed it into the classifier. Or simply rotate your candidate to a certain degree (e.g. for every 10 degree) and then feed them into your classifier and pick the result with the highest confidence.
If you can find the center of the text then you can apply a polar transform to get a feature where rotations in the input become translations in the feature space. That might be easier to work with.
If you can choose what type of symbol you are using as a marker and it doesn't have to be a letter, consider using a rotationally invariant symbol. You can make lots of patterns from rings and circles that would appear the same from every rotation angle.
Training neural nets relies on representing your image with appropriate features. If your labeling is binary (which you could enforce), I would recommend using invariant Fourier descriptors or Zernike moments, which could be made invariant to rotation or scale changes. Simply put, train your classifiers with these features and you will inherently recognize rotated text. Note however that, full rotation invariance is a strong assumptions and would cause you to lose discriminative power e.g. '3' vs 'E'.