Hand Gesture
Classification
Using
Depth Information
Aviad Barzilai & Adi Fuchs - Supervisor Michael Kolomenkin
CGM Laboratory, Department of Electrical Engineering, Technion
Introduction
Interaction with personal computers has become one of the most common and trivial tasks in our everyday life.
But despite the huge advances in computer hardware, the interaction itself has not changed much since the invention of the mouse.
In this project, we develop a much more intuitive system for human-computer interaction. The system uses hand gestures to recognize the human input.
While there is a variety of applications for such a system, ranging from computer games to electronic artwork, we chose to focus on sign language recognition. However we did not neglect the vast applications possible, as our demos will show.
The field of hand gesture recognition has been the topic of many research papers over the past years yet there is still no solid solution on the market. Identifying gestures requires good separation from their background and, more importantly, some perception of depth (to identify obstructing fingers, hidden palms and so on). Attempts to extract this information out of RGB images have failed to yield satisfying results.
In our project we employ a new technology to solve the aforementioned issue. We use an innovative range + video sensor developed by 3DV Systems. The sensors provide the standard RGB video information with additional range information from the camera to the observed objects. The range data allows us to considerably enhance the identification of separate fingers, which is almost impossible with the standard RGB technologies.
Our project is mostly driven by the research aspect of the problem at hand. We have implemented various existing classification methods and have structured and improved them using depth images. Our motivation is building a strong classification algorithm using the additional depth data supplied by the 3DV camera.
We have researched PCA classification, Neural Networks, SIFT feature extraction, AdaBoost with various possible features (both visual and statistical), histogram analysis and general comparison techniques.
Our techniques are novel and may open new directions in the field of gesture classification. Though our goal is to provide a good hand gesture classification technique, our algorithms are general and robust and will prove fruitful for other classification areas where depth holds important information.