Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware

Konrad Moren; Thomas Perschke

doi:10.1117/12.2325391

Translator Disclaimer

8 October 2018 Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware

Konrad Moren, Thomas Perschke

Author Affiliations +

Proceedings Volume 10802, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II; 108020Q (2018) https://doi.org/10.1117/12.2325391
Event: SPIE Security + Defence, 2018, Berlin, Germany

Abstract

Fusion of thermal infrared and visual images is an important technique for real-time surveillance applications. Since image fusion is used in many real-time night vision applications such as target detection, recognition and tracking, it is important to understand the processing requirements and provide computationally efficient methods. In this paper, we present a real-time image fusion system designed for night vision supervisory and monitoring purposes. The system is equipped with two image sensors: TV and IR (thermal infrared). We implement a processing pipeline on the NVIDIA Tegra TX2. The TX2 platform is equipped with a many-core NVIDIA GPU and multi-core ARM CPU. Additionally, we present the system architecture as well as the design process of the efficient, real-time multi-spectral signal-processing algorithm. The algorithm is based on the second-generation wavelets also called lifting scheme. We show also a novel parallelization approach to perform the calculations in place, so no auxiliary memory is needed. This enables a fast parallel and pipelined processing flow. We achieve a considerable speedup compared to an optimized CPU implementation. The experimental results show that the fusion system can realize real-time image fusion processing for dual channels images at the rate of 30 frames per second for the Full-HD images.

Citation Download Citation

Konrad Moren and Thomas Perschke "Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware", Proc. SPIE 10802, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II, 108020Q (8 October 2018); https://doi.org/10.1117/12.2325391

ACCESS THE FULL ARTICLE