Joining geometric and RGB features for RGB-D semantic segmentation

Shaopeng Zhang; Ming Zhong; Gang Zeng; Rui Gan

doi:10.1117/12.2541645

27 November 2019 Joining geometric and RGB features for RGB-D semantic segmentation

Shaopeng Zhang, Ming Zhong, Gang Zeng, Rui Gan

Proceedings Volume 11321, 2019 International Conference on Image and Video Processing, and Artificial Intelligence; 113211F (2019) https://doi.org/10.1117/12.2541645
Event: The Second International Conference on Image, Video Processing and Artifical Intelligence, 2019, Shanghai, China

Abstract

Depth map is a regular format of geometric data structure. Some approaches attempted to harness point cloud from depth channel to extract 3D features and demonstrated the superiority over traditional 2.5D representation approaches. However, how to add 3D features to the pixel on RGB frames in order to incorporate geometric information is a challenging task. In this paper, we propose a simple and general framework combining geometric information of depth maps and RGB information of color maps for semantic segmentation task. Specifically, we first extract geometric feature from an associated point cloud which is harnessed from depth map, and then the RGB feature from color map. Due to the regular format of depth map, the geometric feature can be easily mapped to the corresponding pixel on RGB feature. After that, we get a combination of RGB and geometric features with our Element-Max-Min-Fuse function. Additionally, an efficient decoder module is designed to refine the segmentation results. We demonstrate the effectiveness of the proposed model on S3DIS dataset, the experimental results show that our method enhances the result of using RGB image or point cloud alone.

Citation Download Citation

Shaopeng Zhang, Ming Zhong, Gang Zeng, and Rui Gan "Joining geometric and RGB features for RGB-D semantic segmentation", Proc. SPIE 11321, 2019 International Conference on Image and Video Processing, and Artificial Intelligence, 113211F (27 November 2019); https://doi.org/10.1117/12.2541645

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

RGB color model

Clouds

Image segmentation

3D modeling

Feature extraction

Network architectures

Convolution

Show All Keywords

Keywords/Phrases

Search In:

Publication Years