Another Term work done by me Vivek and Jishnu for our M Tech degree was hand gesture recognition using Matlab. I hope you have already read about one of my previous projects Glove Mouse, which was a gesture control device for PCs, done as part of my B Tech. Well, this time it is just gesture detection, or rather detection/classification of hand gesture/sign images using Matlab. This work is an attempt to implement the concepts discussed in this paper.
This time, the system uses the inbuilt webcam of our laptops rather than using additional hardware. The coding was done in Matlab. The project uses one of the machine learning algorithms, Support Vector Machines(SVM), to classify the images given as the inputs. SVMs are statistical algorithms that finds a pattern in the given inputs and classifies them into separate spaces called hyper planes. Like in any machine learning process, SVM should first be trained, to make it classify the input images. Thus, A set of images are first fed to the machine with their corresponding classification. From this training set, the SVM learns the pattern and applies it to the input images.
Matlab has inbuilt SVM training and classification functions. But the problem is that matlab has only binary SVM, which can classify between two different vectors only, in this case only two different gestures. Hence further processing is required to classify more than two gestures and this is done by classifying the input to two different vectors at a time, thus repeating this process to cover all the classes and the class to which the image is classified the most is taken as the final result.
The input to train and classify the SVM is the feature vector of the input images. To extract the features all the pictures are re sized and the feature extraction is done using 7 levels of wavelet decomposition. To make the process simpler, the re sized images are converted into binary image (or simply black and white image) and then wavelet transform using a Haar Wavelet is applied. The resulting coefficient matrix is used as the input to the SVM.
The code developed was able to classify most of the images but the quality of the camera and the backgrounds made the classification a challenging one. Hence there were several wrong classifications when it was run. Some 25 images for each of the 3 gestures(victory, thumbs up and high five) were used to train the SVM. The screenshot shows the result of running the code with 9 test images. Of them 2 gestures were classified wrongly. The matlab code can be found here.
This is not a real time gesture detection/ recognition. Hence it can only used for applications such as sign language translation or such kinds of applications were the pictures are taken beforehand. When made into a real time gesture recognition, the applications are vast such as gesture control, real time sign language translation, etc.