RGB-D action recognition based on discriminative common structure learning model

Tianshan Liu; Jun Kong; Min Jiang; Hongtao Huo

doi:10.1117/1.JEI.28.2.023012

13 March 2019 RGB-D action recognition based on discriminative common structure learning model

Tianshan Liu, Jun Kong, Min Jiang, Hongtao Huo

Author Affiliations +

Funded by: National Natural Science Foundation of China (NSFC), Natural Science Foundation of China, China Postdoctoral Science Foundation, Jiangsu Postdoctoral Science Foundation, Scientific and Technological Aid Program of Xinjiang, Postgraduate Research and Practice Innovation Program of Jiangsu Province

Journal of Electronic Imaging, Vol. 28, Issue 2, 023012 (March 2019). https://doi.org/10.1117/1.JEI.28.2.023012

Abstract

The emergence of low-cost depth cameras creates potential for RGB-D based human action recognition. However, most of the existing RGB-D based approaches simply concatenate original heterogeneous features without discovering the latent relations among different modalities. We propose a discriminative common structure learning (DCSL) model for human action recognition from RGB-D sequences. Specifically, we extract deep learning-based features and hand-crafted features from multimodal data (skeleton, depth, and RGB). In particular, we propose a deep architecture based on 3-D convolutional neural network to automatically extract deep spatiotemporal features from raw sequences. The proposed DCSL model utilizes a generalized version of collective matrix factorization to learn shared features among different modalities. To perform supervised learning and preserve intermodal similarity, we formulate a graph regularization term by considering both label information and similar geometric structure of multimodal data, which intends to improve the discriminative power of shared features. Moreover, we solve the objective function using an iterative optimization algorithm. Then, an improved collaborative representation classifier is employed to perform computationally efficient action recognition. Experimental results on four action datasets demonstrate the superior performance of the proposed method.

Citation Download Citation

Tianshan Liu, Jun Kong, Min Jiang, and Hongtao Huo "RGB-D action recognition based on discriminative common structure learning model," Journal of Electronic Imaging 28(2), 023012 (13 March 2019). https://doi.org/10.1117/1.JEI.28.2.023012

Received: 18 October 2018; Accepted: 26 February 2019; Published: 13 March 2019

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE