Appearance-based human gesture recognition using multimodal features for human computer interaction

Dan Luo; Hua Gao; Hazim Kemal Ekenel; Jun Ohya

doi:10.1117/12.872525

2 February 2011 Appearance-based human gesture recognition using multimodal features for human computer interaction

Dan Luo, Hua Gao, Hazim Kemal Ekenel, Jun Ohya

Proceedings Volume 7865, Human Vision and Electronic Imaging XVI; 786509 (2011) https://doi.org/10.1117/12.872525
Event: IS&T/SPIE Electronic Imaging, 2011, San Francisco Airport, California, United States

Abstract

The use of gesture as a natural interface plays an utmost important role for achieving intelligent Human Computer Interaction (HCI). Human gestures include different components of visual actions such as motion of hands, facial expression, and torso, to convey meaning. So far, in the field of gesture recognition, most previous works have focused on the manual component of gestures. In this paper, we present an appearance-based multimodal gesture recognition framework, which combines the different groups of features such as facial expression features and hand motion features which are extracted from image frames captured by a single web camera. We refer 12 classes of human gestures with facial expression including neutral, negative and positive meanings from American Sign Languages (ASL). We combine the features in two levels by employing two fusion strategies. At the feature level, an early feature combination can be performed by concatenating and weighting different feature groups, and LDA is used to choose the most discriminative elements by projecting the feature on a discriminative expression space. The second strategy is applied on decision level. Weighted decisions from single modalities are fused in a later stage. A condensation-based algorithm is adopted for classification. We collected a data set with three to seven recording sessions and conducted experiments with the combination techniques. Experimental results showed that facial analysis improve hand gesture recognition, decision level fusion performs better than feature level fusion.

Citation Download Citation

Dan Luo, Hua Gao, Hazim Kemal Ekenel, and Jun Ohya "Appearance-based human gesture recognition using multimodal features for human computer interaction", Proc. SPIE 7865, Human Vision and Electronic Imaging XVI, 786509 (2 February 2011); https://doi.org/10.1117/12.872525

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available