Unsupervised synthetic image refinement via contrastive learning and consistent semantic-structural constraints

Ganning Zhao; Tingwei Shen; Suya You; C.-C. Jay Kuo

doi:10.1117/12.2663897

12 June 2023 Unsupervised synthetic image refinement via contrastive learning and consistent semantic-structural constraints

Ganning Zhao, Tingwei Shen, Suya You, C.-C. Jay Kuo

Author Affiliations +

Proceedings Volume 12538, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V; 125381M (2023) https://doi.org/10.1117/12.2663897
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Conference Poster

Abstract

Ensuring the realism of computer-generated synthetic images is crucial to deep neural network (DNN) training. Due to different semantic distributions between synthetic and real-world captured datasets, there exists semantic mismatch between synthetic and refined images, which in turn results in the semantic distortion. Recently, contrastive learning (CL) has been successfully used to pull correlated patches together and push uncorrelated ones apart. In this work, we exploit semantic and structural consistency between synthetic and refined images and adopt CL to reduce the semantic distortion. Besides, we incorporate hard negative mining to improve the performance furthermore. We compare the performance of our method with several other benchmarking methods using qualitative and quantitative measures and show that our method offers the state-of-the-art performance.

Citation Download Citation

Ganning Zhao, Tingwei Shen, Suya You, and C.-C. Jay Kuo "Unsupervised synthetic image refinement via contrastive learning and consistent semantic-structural constraints", Proc. SPIE 12538, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V, 125381M (12 June 2023); https://doi.org/10.1117/12.2663897

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available