Deep network based 3D hand keypoints prediction from single RGB images

Jialong Wang; Nong Sang

doi:10.1117/12.2522496

13 May 2019 Deep network based 3D hand keypoints prediction from single RGB images

Jialong Wang, Nong Sang

Proceedings Volume 10995, Pattern Recognition and Tracking XXX; 109950V (2019) https://doi.org/10.1117/12.2522496
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

3D hand keypoints prediction is an important and fundamental task in Human-Computer Interaction. In this paper, we present an approach to predict 3D hand keypoints from single RGB images. Single RGB images are very common in daily life. However, it is challenging to predict 3D hand keypoints using single RGB images, because of depth ambiguities and occlusions. To deal with these challenges, we exploit deep neural networks to predict 3D hand keypoints. So far, there are several methods which predict 3D hand keypoints from single RGB images. Most of them separate the task into three stages. i.e., hand detection, 2D hand keypoints estimation and 3D hand keypoints prediction. We follow the idea and focus on the 2D hand keypoints estimation and 3D hand keypoints prediction. We improve an existing deep-network-based technique and get better results. Specifically, we combine the convolution and deconvolution network to get the pixel-wise estimation of 2D hand keypoints, and propose a new loss function to predict 3D hand keypoints from 2D keypoints. We evaluate our network on several public datasets and get better results than several other methods. Besides, ablation studies demonstrate that our network is valid.

Citation Download Citation

Jialong Wang and Nong Sang "Deep network based 3D hand keypoints prediction from single RGB images", Proc. SPIE 10995, Pattern Recognition and Tracking XXX, 109950V (13 May 2019); https://doi.org/10.1117/12.2522496

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available