I am currently working on a gesture recognition system (for an Android Application). I think that I have completed the Image processing stage, where I am able to extract the contour of the hand (I am wearing a glove to avoid background subtraction for now).
I am also getting the bounding ellipse/rectangle, the centroid as 'important' shape features of the hand.
My problem is that not a lot of literature indicate what the next stage is before the classification of the image through the hidden markov model. I am finding this 'Feature Extraction' stage very ambiguous.
Currently I am getting a list of angles of the contour (which is approximated so as to obtain a limited number of angles)
The problem is that I am clueless as what to do next. When I search for literature as 'Shape classification using HMMs' I still find it hard to what should I do next.
As a tool for HMM I plan to use JaHMM. But I don't know how to experiment with the tool at this stage since I do not know what will be the input to this classification stage!
I have encountered a list of functions I found in some literature, such as Fast Fourier Descriptors, Curvature Descriptors, B Spline; yet I am clueless as to how to apply these functions to my current data (i.e. list of angles, e.g.: -63, 154, 3, 23, 54, ....)
UPDATE 1:
Thank you for your information. @Peter K.
Regarding the poses: I was going to go for a set of words in American Sign Language which are fingerspelled, e.g 'dog' -> 'd' 'o' 'g' (3 states); but the moment I haven't decided what to go for.
I am currently reading some more papers to see what type of information I should extract, such as :
- centroid of hand
- angle of motion
- distance from a particular point to different points of the hand contour (etc..)
Now I have encountered a paper which seems to show what I want to do, I'm not sure:
http://espace.library.uq.edu.au/eserv.php?pid=UQ:10700&dsID=n0273.pdf
I am currently reading section 5 - Vector Quantization (I heard of this term before but do not actually know what it involves, and in figure 5 there seems to be a complex algorithm which, if I understood correcty, converts the set of values I achieve from the hand (just mentioned) into 1 digit which I can use as the Observation sequence to train an HMM for 1 particular sign. Do you think I'm moving on the right track? (I'm working on Android, (NDK), I found JaHMM as an HMM tool, and using OpenCV for image processing.
@Peter K. Thank you for your answer. Regarding the data generation I was planning to follow the steps of this paper, after I produce my personal dataset: (section 4 and 5) http://www.i.ci.ritsumei.ac.jp/~shimada/papers/vi02-tanibata.pdf
UPDATE 2: keeping in mind that a gesture is formed of {posture a, posture b, posture c}
I am now thinking that I must make use of some form of classification algorithm. That is, I currently have a set of feature vectors:
Posture A: [angle of ellipse surrounding it, height:width ratio]
- 0.802985 33.909615
- 0.722824 31.209663
- 0.734535 30.206722
- 0.68397 31.838253
- 0.713706 34.29641
- 0.688798 30.603661
- 0.721395 34.880161
Posture B: [structured the same as posture A]
- 0.474164 16.077467
- 0.483104 14.526289
- 0.478904 14.800572
- 0.483134 14.523611
- 0.480608 14.41159
- 0.481552 15.563665
- 0.497951 15.563585
etc..
and I would like that when I feed a feature vector I obtain a simple symbol, e.g. 'A', 'B', etc.
Is this possible? I also migrated the question here: https://stackoverflow.com/questions/15602963/vector-quantization-algorithms-used-to-provide-observation-sequences-for-hidden