Transfer learning represents a recent paradigm shift in the way we build artificial intelligence (AI) systems. In contrast to training task-specific models, transfer learning involves pre-training deep learning models on a large corpus of data and minimally fine-tuning them for adaptation to specific tasks. Even so, for 3D medical imaging tasks, we do not know if it is best to pre-train models on natural images, medical images, or even synthetically generated MRI scans or video data. To evaluate these alternatives, here we benchmarked vision transformers (ViTs) and convolutional neural networks (CNNs), initialized with varied upstream pre-training approaches. These methods were then adapted to three unique downstream neuroimaging tasks with a range of difficulty: Alzheimer's disease (AD) and Parkinson’s disease (PD) classification, “brain age” prediction. Experimental tests led to the following key observations: 1. Pre-training improved performance across all tasks including a boost of 7.5% for AD classification and 4.5% for PD classification for the ViT and 19.1% for PD classification and reduction in brain age prediction error by 1.26 years for CNNs, 2. Pre-training on large-scale video or synthetic MRI data boosted performance of ViTs, 3. CNNs were robust in limited-data settings, and in-domain pretraining enhanced their performances, 4. Pre-training improved generalization to out-of-distribution datasets and sites. Overall, we benchmarked different vision architectures, revealing the impact of pre-training them with emerging datasets for model initialization. The resulting pre-trained models can be adapted to a range of downstream neuroimaging tasks, especially when training data for a domain-specific target task is limited.
KEYWORDS: Brain, Magnetic resonance imaging, Data modeling, Education and training, Dementia, 3D modeling, Neuroimaging, Diffusion, Diffusion weighted imaging, Deep learning
Machine learning methods have been used for over a decade for staging and subtyping a variety of brain diseases, offering fast and objective methods to classify neurodegenerative diseases such as Alzheimer’s disease (AD). Deep learning models based on convolutional neural networks (CNNs) have also been used to infer dementia severity and predict future clinical decline. Most CNN-based deep learning models use T1-weighted brain MRI scans to identify predictive features for these tasks. In contrast, we examine the added value of diffusion-weighted MRI (dMRI) - a variant of MRI, sensitive to microstructural tissue properties - as an additional input in CNN-based models of dementia severity. dMRI is sensitive to microstructural brain abnormalities not evident on standard anatomical MRI. By training CNNs on combined anatomical and diffusion MRI, we hypothesize that we could boost performance when predicting widely-used clinical assessments of dementia severity, such as individuals’ scores on the ADAS11, ADAS13, and MMSE (mini-mental state exam) clinical scales. For benchmarking, we evaluate CNNs that use T1-weighted MRI and dMRI to estimate “brain age” - the task of predicting a person’s chronological age from their neuroimaging data. To assess which dMRI-derived maps were most beneficial, we computed DWI-derived diffusion tensor imaging (DTI) maps of mean and radial diffusivity (MD/RD), axial diffusivity (AD) and fractional anisotropy (FA) for 1198 elderly subjects (age: 74.35 +/- 7.74 yrs.; 600 F/598 M, with a distribution of 636 CN/421 MCI/141 AD) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). We tested both 2D Slice CNN and 3D CNN neural network models for the above predictive tasks. Our results suggest that for at least some deep learning architectures, diffusion-weighted MRI may enhance performance for several AD-relevant deep learning tasks relative to using T1-weighted images alone.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.