A multimodal vision sensor for autonomous driving

Dongming Sun; Xiao Huang; Kailun Yang

doi:10.1117/12.2535552

7 October 2019 A multimodal vision sensor for autonomous driving

Dongming Sun, Xiao Huang, Kailun Yang

Proceedings Volume 11166, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III; 111660L (2019) https://doi.org/10.1117/12.2535552
Event: SPIE Security + Defence, 2019, Strasbourg, France

Abstract

This paper describes a multimodal vision sensor that integrates three types of cameras, including a stereo camera, a polarization camera and a panoramic camera. Each sensor provides a specific dimension of information: the stereo camera measures depth per pixel, the polarization obtains the degree of polarization, and the panoramic camera captures a 360° landscape. Data fusion and advanced environment perception could be built upon the combination of sensors. Designed especially for autonomous driving, this vision sensor is shipped with a robust semantic segmentation network. In addition, we demonstrate how cross-modal enhancement could be achieved by registering the color image and the polarization image. An example of water hazard detection is given. To prove the multimodal vision sensor’s compatibility with different devices, a brief runtime performance analysis is carried out.

Conference Presentation

Citation Download Citation

Dongming Sun, Xiao Huang, and Kailun Yang "A multimodal vision sensor for autonomous driving", Proc. SPIE 11166, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III, 111660L (7 October 2019); https://doi.org/10.1117/12.2535552

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available