Tracking a user's face and hands in three-dimensions in real-time

Corey Manders; Farzam Farbiz; Ishtiaq Rasool Khan; Miaolong Yuan; Ka Yin Tang; Mei Hwan Loke; Gim Guan Chua

doi:10.1117/12.806097

Translator Disclaimer

2 February 2009 Tracking a user's face and hands in three-dimensions in real-time

Corey Manders, Farzam Farbiz, Ishtiaq Rasool Khan, Miaolong Yuan, Ka Yin Tang, Mei Hwan Loke, Gim Guan Chua

Author Affiliations +

Proceedings Volume 7251, Image Processing: Machine Vision Applications II; 72510L (2009) https://doi.org/10.1117/12.806097
Event: IS&T/SPIE Electronic Imaging, 2009, San Jose, California, United States

Abstract

This work outlines a system in which a stereo camera may effectively track a user's face and hands in three dimensions. Given this information, a method for controlling objects in three dimensions is also described. The system begins by finding faces. If more than one face is found in the image, the algorithm uses depth information to isolate the face that is closest to the camera. The algorithm then gathers information about the user's skin tone by examining the content of the face found. For much of the processing, only the hue and saturation components are used after applying an HSV to RGB transformation given the camera output. The skin tone information in tandem with depth is then used to isolate the user's hands, and track them in three dimensions. To be used as an effective interface, the system uses information of the two hands relative to the user's face. In controlling an object in three dimensions, if the user would like to move the object up, he or she simply positions both hands above his or her face. Similar commands allow the user to apply a translational factor in three dimensions, as well as applying yaw and roll when wanted.

Citation Download Citation

Corey Manders, Farzam Farbiz, Ishtiaq Rasool Khan, Miaolong Yuan, Ka Yin Tang, Mei Hwan Loke, and Gim Guan Chua "Tracking a user's face and hands in three-dimensions in real-time", Proc. SPIE 7251, Image Processing: Machine Vision Applications II, 72510L (2 February 2009); https://doi.org/10.1117/12.806097

ACCESS THE FULL ARTICLE