Performance deterioration of deep neural networks for lesion classification in mammography due to distribution shift: an analysis based on artificially created distribution shift

Alexej Gossmann; Kenny H. Cha; Xudong Sun

doi:10.1117/12.2551346

16 March 2020 Performance deterioration of deep neural networks for lesion classification in mammography due to distribution shift: an analysis based on artificially created distribution shift

Alexej Gossmann, Kenny H. Cha, Xudong Sun

Author Affiliations +

Proceedings Volume 11314, Medical Imaging 2020: Computer-Aided Diagnosis; 1131404 (2020) https://doi.org/10.1117/12.2551346
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

Despite the prominent success of deep learning (DL) in medical imaging for tasks such as computer-aided detection and diagnosis, the field faces a number of challenging problems. An important issue is that of mismatch of data distributions between different data sources, also known as a distribution shift. Distribution shifts may also be present between different subpopulations or subgroups. Distribution shifts that are not easily detectable can prevent the successful deployment of DL models in medical imaging. We use variational inference to create subsets of a given dataset while enforcing artificial distribution shifts between these subsets, thus creating subsets with different characteristics that represent different pseudo “data sources”. By training and testing ROI-based malignant/benign lesion classification models over these pseudo data sources, we evaluate the extent to which distribution shift could deteriorate the performance of popular DL models. We show that distribution shift indeed poses a serious concern for malignant/benign lesion classification in mammography, and we show that the algorithmically created pseudo data sources may not correspond to any recorded clinical or image characteristics. This study shows a potential method for evaluating deep learning algorithms for robustness against distribution shifts. Furthermore, our technique can serve as a benchmark method for development of new models which aim to be robust to distribution shift.

Conference Presentation

Citation Download Citation

Alexej Gossmann, Kenny H. Cha, and Xudong Sun "Performance deterioration of deep neural networks for lesion classification in mammography due to distribution shift: an analysis based on artificially created distribution shift", Proc. SPIE 11314, Medical Imaging 2020: Computer-Aided Diagnosis, 1131404 (16 March 2020); https://doi.org/10.1117/12.2551346

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available