Zero-shot contrastive vision-language pre-training for traffic sign recognition in adverse weather conditions

Xinyu Lu

doi:10.1117/12.3042011

29 August 2024 Zero-shot contrastive vision-language pre-training for traffic sign recognition in adverse weather conditions

Xinyu Lu

Author Affiliations +

Proceedings Volume 13249, International Conference on Computer Vision, Robotics, and Automation Engineering (CRAE 2024); 1324907 (2024) https://doi.org/10.1117/12.3042011
Event: 2024 International Conference on Computer Vision, Robotics and Automation Engineering, 2024, Kunming, China

Abstract

Robust traffic sign detection and recognition under adverse weather conditions is a critical challenge for autonomous driving systems. This paper presents a novel approach that combines zero-shot learning with contrastive vision-language pre-training to enhance the resilience of traffic sign recognition systems against weather-induced visual impairments. Our method leverages a limited dataset to train a model capable of understanding and processing images degraded by various weather conditions such as rain, fog, and snow without direct exposure to these conditions during training. By integrating descriptive language data with visual cues, our model learns to identify and interpret traffic signs through a generalizable semantic embedding, facilitating robust detection and recognition across unseen weather scenarios. The framework employs a two-stage training process: the initial stage focuses on learning general visual features from minimally weather-affected images, while the subsequent stage enhances the model's ability to predict and adapt to weather-specific distortions using a novel zero-shot learning strategy. Experimental evaluations demonstrate superior performance over traditional methods, particularly in zero-shot scenarios where the model encounters completely novel weather conditions. This approach not only advances the field of image restoration in severe weather but also sets a new standard for the deployment of vision-based systems in real-world environments where variable weather is a common challenge.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xinyu Lu "Zero-shot contrastive vision-language pre-training for traffic sign recognition in adverse weather conditions", Proc. SPIE 13249, International Conference on Computer Vision, Robotics, and Automation Engineering (CRAE 2024), 1324907 (29 August 2024); https://doi.org/10.1117/12.3042011

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Visualization

Adverse weather

Data modeling

Education and training

Visual process modeling

Modulation

Rain

Show All Keywords

Keywords/Phrases

Search In:

Publication Years