This PDF file contains the front matter associated with SPIE Proceedings Volume 12467, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Artificial intelligence (AI) and machine learning models are being increasingly deployed in real-world applications including medical imaging. In many of these applications, there is strong motivation to develop hybrid systems in which humans and AI algorithms can work together, leveraging their complementary strengths and weaknesses. In this talk, I will discuss the promises and pitfalls of AI-assisted decision-making where a human decision-maker is aided by an AI. I will present empirical research that investigates the effectiveness of AI-assisted decisions and the cognitive decision process in different paradigms for presenting AI advice. I will also discuss the question of "machine theory of mind" and "theory of machine," how humans and machines can efficiently form mental models of each other to collaborate more effectively.

Satisfaction of Search (SOS), an increased likelihood of missing a lesion/target when another lesion/target was already detected in the same image, is a known search problem in medical image perception and cognitive science. Currently, it is unknown the extent to which SOS affects lesion detection in breast imaging. Here we investigated SOS rates with novices (college undergraduates) and experts (residents, fellows, and certified radiologists) in virtual (computer-generated) tomograms. The virtual images were created with the OpenVCT framework simulating the breast anatomy of patients and could contain zero, one, or two masses. Participants were asked to search and click on up to two masses per image. When compared to novices, experts: 1) made fewer false alarms (experts 19.5%, novices 29.1%; marginally significant effect), 2) were better in correct rejections (experts 76.7%, novices 44.5%), and 3) made fewer SOS errors (experts 14.2%, novices 30.7%). These findings are the first to demonstrate the SOS effect in breast imaging and emphasize the importance of expertise in breast cancer detection.

Detection of low contrast liver metastases varies between radiologists. Training may improve performance for lower-performing readers and reduce inter-radiologist variability. We recruited 31 radiologists (15 trainees, eight non-abdominal staff, and eight abdominal staff) to participate in four separate reading sessions: pre-test, search training, classification training, and post-test. In the pre-test, each radiologist interpreted 40 liver CT exams containing 91 metastases, circumscribed suspected hepatic metastases while under eye tracker observation, and rated confidence. In search training, radiologists interpreted a separate set of 30 liver CT exams while receiving eye tracker feedback and after coaching to increase use of coronal reformations, interpretation time, and use of liver windows. In classification training, radiologists interpreted up to 100 liver CT image patches, most with benign or malignant lesions, and compared their annotations to ground truth. Post-test was identical to pre-test. Between pre- and post-test, sensitivity increased by 2.8% (p = 0.01) but AUC did not change significantly. Missed metastases were classified as search errors (<2 seconds gaze time) or classification errors (>2 seconds gaze time) using the eye tracker. Out of 2775 possible detections, search errors decreased (10.8% to 8.1%; p < 0.01) but classification errors were unchanged (5.7% vs 5.7%). When stratified by difficulty, easier metastases showed larger reductions in search errors: for metastases with average sensitivity of 0-50%, 50-90%, and 90-100%, reductions in search errors were 16%, 35%, and 58%, respectively. The training program studied here may be able to improve radiologist performance by reducing errors but not classification errors.

Humans process the visual world with varying resolution across the visual field and sample information using eye movements to point the high-resolution central vision (fovea) to regions of interest. What eye movements optimize decisions in tasks such as target detection and localization? In 2005, Najemnik & Geisler proposed the Bayesian Ideal searcher (IS) that takes into account the foveated properties of human visual and employs the optimal fixation selection strategy to maximize visual search performance. In addition, they proposed a computationally simpler model, entropy limit minimization (ELM), that approximates the IS searcher. One limitation of these models is that they were developed with the assumption that the visual system’s internal responses across fixations are statistically independent. This assumption will not hold for search tasks such as in 2D medical images for which the external noise and anatomical noise results in correlations in internal responses across saccadic fixations (inter-saccade response correlations). In this work, we present image-computable foveated IS and ELM models that accommodate inter-saccade response correlations. We demonstrate that for static images, the optimal searchers that account for the inter-saccade correlations (i.e., IS-COR and ELM-COR) significantly outperform the traditional methods that ignore such correlations. Moreover, the novel ELM-COR achieves a similar performance as the IS-COR but runs about 18 times faster. Together, the IS-COR and ELM-COR extend the optimal searcher framework to evaulate human fixations for more realistic search tasks with inter-saccadic correlations.

Digital Pathology (DP) reporting workstations permit eye tracking experiments which can aid our understanding of reading strategies and medical errors in pathology. However, eye tracking with DP slides is complex due to the nature of the slide viewing process: slide panning and zooming. Eye tracking technology records gaze coordinates to a screen surface, but these coordinates do not account for the ever-changing on-screen content (due to slide navigation), and therefore it is essential to track pathologists’ slide navigation to determine where on the slide the pathologist has viewed and what features were fixated. Additionally, visualising the resulting eye tracking data proves challenging due to the zooming component. Other eye tracking studies in DP have accounted for slide navigation by employing custom slide viewers that output slide movements as a data stream with the eye tracking data which are co-registered for analysis. We are conducting a DP eye tracking study using a commercial slide viewer which has been adopted at selected UK hospital sites, but slide movement data cannot be outputted as a data stream in this context. Therefore, we’re developing a software platform using computer vision techniques that can be applied to the recorded screen capture of the DP workstation which is synchronised with the eye tracking data. The developed algorithm could be adapted for use with other commercial slide viewers for future studies. Here, we explore how studies have addressed these issues and we discuss our approach.

Digital Breast Tomosynthesis (DBT) increases breast cancer detection rates but produces a significantly greater number of images for screeners to read compared to traditional two-dimensional (2-D) mammograms. Putting screeners at risk of fatigue and therefore error in detecting cancers.
The aim of this study was to explore if screeners showed differences in subjective fatigue, blink metrics and diagnostic accuracy during a DBT reading session with and without breaks.
Prospective study including 45 participants from 6 different hospital sites around England between December 2020 to April 2022. Non-intrusive, screen mounted eye tracking cameras (60Hz sampling rate) were set up in the participant’s natural reading environment. Forty DBT cases were read in a random order (47.5% malignant, 12.5% benign, 40% normal). Each breast was rated as normal or benign (return to screen) or indeterminate, suspicious or highly suspicious (recall). Twenty-one participants had a break at approximately 40 minutes into the session.
Participants without a break showed a significantly greater difference in subjective fatigue before and after the reporting session (44% vs 33%, p=0.037). Furthermore, those without breaks exhibited significantly greater blinks per minute (15.75 vs 13.25, p<0.001) and blink duration (milliseconds) (296 vs 286, p<0.001). There was no significant difference in overall accuracy between the cohorts (p=0.921).
Blink metrics have the potential to be used in identifying early onset of fatigue during reading sessions.

This study investigated whether a global radiomic signature (i.e., a set of global radiomic features) from mammograms can predict radiologists’ difficult-to-interpret normal cases. Retrospective non-identifiable data collected from 342 radiologists interpreting 81 normal mammograms were used to group cases as difficult-to-interpret (41 cases) and easy-to-interpret (40 cases) based on one-third of cases having the correspondingly highest and lowest difficulty scores. A set of 34 global radiomic features per image were extracted based on regions of interests delineated using lattice- and squared-based approaches, and normalised. Three machine learning classification models were constructed: 1). CC, using the 34 global radiomic features derived from craniocaudal images only, and 2). MLO, using the features from mediolateral oblique images only, both based on a random forest method for differentiating difficult-to-interpret from easy-to-interpret normal cases, and 3). CC+MLO model using the median predictive scores from both CC and MLO models. We trained and validated the models using leave-one-out-cross-validation approach. Performances of the models were measured by the area under the receiver operating characteristic curve (AUC). The CC+MLO model outperformed (0.73 AUC, 0.62 to 0.83) the CC (0.70 AUC, 0.62 to 0.78) and MLO (0.68 AUC, 0.60 to 0.76) models. The results showed that the global mammographic radiomic signature has the ability to predict radiologists’ difficult-to-interpret normal cases.

The final stage in the medical imaging diagnostic system is the radiologist’s interpretation of the images, though research on the factors influencing performance in digital breast tomosynthesis (DBT) is inconclusive. This study seeks to understand the performance of radiologists in reading DBT images and the parameters impacting observer performance in three different countries. The study used a DBT mammogram test to compare the performance of radiologists from Australia, China and Iran in reading thirty-five DBT cases. A range of performance metrics including specificity, sensitivity, lesion sensitivity, ROC AUC and JAFROC FOM were generated for each radiologist upon the conclusion of the test set. The radiologists also provided demographic information relating to their experience in reading digital mammograms and DBT. Each country had a greater percentage of radiologists that have completed a breast imaging fellowship compared to those that have not. Australia had a greater percentage of radiologists that have completed training in DBT reading (Australia=88.2%), while China and Iran had a smaller percentage of radiologists that have not completed training in DBT reading (China=37%, Iran=40%). Significant differences were identified between the three countries in specificity (p=.001), lesion sensitivity (p=.016), ROC (p<.001) and JAFROC (p<.001). Australia had the highest mean value for all performance metrics, while China had the lowest mean value for all performance metrics. Australian radiologists have a moderate positive correlation between lesion sensitivity and the number of years reading DBT images (r=.513, p=.042). Iranian radiologists who read more than 20 DBT cases per week obtained significantly higher performance in lesion sensitivity 73.3% vs. 51.8%; p=.032) than the ones who read less than 20 DBT cases per week.

The global radiomic signature extracted from mammograms can indicate that malignancy appearances are present within an image. This study focuses on a set of 129 screen-detected breast malignancies, which were also visible on the prior screening examinations (i.e., missed cancers based on the priors). All cancer signs on the prior examinations were actionable based on the opinion of a panel of three experienced radiologists, who retrospectively interpreted the prior examinations (knowing that a later screening round had revealed a cancer). We investigated if the global radiomic signature could differentiate between screening rounds: when the cancer was detected (“identified cancers”), from the round immediately before (“missed cancers”). Both identified cancers and “missed cancers” were collected using a single vendor technology. A set of “normals”, matched based on mammography units, was also retrieved from a screening archive. We extracted a global radiomic signature, containing first and second-order statistics features. Three classification tasks were considered: (1) “identified cancers” vs “missed cancers”, (2) “identified cancers” vs “normals”, (3) “missed cancers” vs “normal”. To train and validate the models, leave-one-case-out cross-validation was used. The classifier resulted in an AUC of 0.66 (95%CI=0.60-0.73, P<0.05) for “missed cancers” vs “identified cancers” and an AUC of 0.65 (95%CI=0.60-0.69, P<0.05) for “normals” vs “identified cancers”. However, the AUC of the classifier for differentiating “normals” from “missed cancers” was at chance-level (AUC=0.53 (95%CI=0.48-0.58, P=0.23). Therefore, eliminating some of these “missed” cancers in clinical practice would be very challenging as the global signal of the malignancy that help with a diagnosis, are at best weak.

Previous studies reported that the cancer subtypes radiologists struggling to detect successfully varied across countries in mammography interpretation. However, little is known whether such variation is also in radiologists’ perception of local cancer-free areas. This study compared the cancer-free areas incorrectly flagged as cancer by radiologists from two populations in reading dense screening mammograms. We collected reading data from 20 Chinese and 16 Australian radiologists who previously evaluated 60 dense screening cases. For each cohort, findings from all readers were pooled together, and the local cancer-free areas classified as cancer were identified. Particularly the areas misclassified by readers from both cohorts were recognized and displayed on the mammograms as overlaps. For each overlap, we counted the error rate, the proportion of readers failing to distinguish between normality and abnormality, as a measure of the actual difficulty level for each reader cohort. Afterward, the Spearman correlation was performed to explore whether the calculated cohort-specific difficulty levels were correlated. A similar analysis was conducted on two geographically-distant groups within China. Results showed that between Chinese and Australian radiologists, the correlation was only found in the cancer-free views of cancer cases (r=0.902, p=0.004). However, between the two groups within China, we found strong correlations in both cancer-containing (r=0.833, p=0.333) and cancer-free views (r=0.955, p=0.022) of cancer cases, despite an insignificant correlation in normal cases. In conclusion, radiologists from different populations display various error-making patterns in reading dense screening mammograms, while those with similar demographic characteristics share the diagnosis to a certain degree.

The ‘aging’ of artificial intelligence/machine learning (AI/ML) models after initial development and evaluation is known to frequently occur and can pose substantial problems. When there are changes in population, disease characteristics, imaging equipment, or protocols, model performance may start to deteriorate, and the performance predicted in a research setting may no longer hold after deployment (either in a clinical setting or in further research). This data shift phenomenon is a common problem in AI/ML.
We trained and evaluated a previously in-house developed AI/ML model for COVID severity prediction using two COVID-19-positive consecutive adult patient cohorts from a single institution. The first cohort was from the time that the Delta strain was dominant accounting for <95% of cases (June 24-December 11, 2021, 820 patients, 1331 chest radiographs (CXRs)) and the second cohort was from the time that the Omicron variant was dominant (Jan 1-21, 2022, 656 patients, 970 CXRs). Inclusion criteria were COVID-positivity and the availability of CXR imaging exams, in general for patients not admitted to ICU and prior to ICU admission for those patients admitted to ICU as part of their treatment. Exclusion criteria were image acquisition in ICU or the presence of mechanical ventilation. Our image-based AI/ML model was trained to predict, based on each frontal CXR from a COVID-positive patient, whether this patient would be admitted to ICU within a 24, 48, 72, or 96-hour window. The model was evaluated 1) in a cross-sectional test when trained on a subset/tested on an independent subset of the Delta cohort, 2) similarly for the Omicron cohort, and 3) in a longitudinal test when trained on the Delta cohort/tested on the Omicron cohort.
Cohorts were similar in ICU admission rate and fraction of portable CXRs, while immunization rate was higher for the Omicron cohort. The model did not demonstrate signs of aging with performances in the longitudinal test being very similar to those within the Delta cohort, e.g., an area under the ROC curve in the task of predicting ICU admission within 24 hours of 0.76 [0.68; 0.84] when trained/tested within the Delta cohort and 0.77 [0.73; 0.80] for the longitudinal test (p>0.05). The performance within the Omicron cohort was similar as well, at 0.76 [0.66; 0.84].
Our AI/ML model for COVID-severity prediction did not demonstrate signs of aging in a longitudinal test when trained on the Delta cohort and applied as-is to the Omicron cohort.

Chest x-ray radiography (CXR) is widely used in screening and detecting lung diseases. However, reading CXR images is often difficult resulting in diagnostic errors and inter-reader variability. To address this clinical challenge, a Multi-task, Optimal-recommendation, and Max-predictive Classification and Segmentation (MOM-ClaSeg) system is developed to detect and delineate different abnormal regions of interest (ROIs) on CXR, make multiple recommendations of abnormalities sorted by the generated probability scores, and automatically generate diagnostic report. MOM-ClaSeg consists of convolutional neural networks to generate a detection, finer-grained segmentation and prediction score for each ROI based on augmented MaskRCNN framework, and multi-layer perception neural networks to fuse results to generate the optimal recommendation for each detected ROI based on decision fusion framework. Total of 310,333 adult CXR containing 67,071 normal and 243,262 abnormal images depicting 307,415 confirmed ROIs of 65 different abnormalities were assembled as to train MOM-ClaSeg. An independent 22,642 CXR was assembled to test MOMClaSeg. Radiologists detected 6,646 ROIs that depict 43 different types of abnormalities on 4,068 CXR images. Comparing with radiologists’ detection results, MOM-ClaSeg system detected 6,009 true-positive ROIs and 6,379 false-positive ROIs, which represents 90.3% sensitivity and 0.28 false-positive ROIs per image. For the eight common diseases, the computed areas under ROC curves ranged from 0.880 to 0.988. Additionally, 70.4% of MOM-ClaSeg system-detected abnormalities along with system-generated diagnostic reports were directly accepted by radiologists. This study presents the first AI-based multi-task prediction system to detect different abnormalities and generate diagnostic reports to assist radiologists accurately and/or efficiently detecting lung diseases.

MIDRC was created to facilitate machine learning research for tasks including early detection, diagnosis, prognosis, and assessment of treatment response related to the COVID-19 pandemic and beyond. The purpose of the Technology Development Project (TDP) 3c is to create resources to assist researchers in evaluating the performance of their machine learning algorithms. An interactive decision tree has been developed, organized by the type of task that the machine learning algorithm is being trained to perform. The user can select information such as: (a) the type of task, (b) the nature of the reference standard, and (c) the type of the algorithm output. Based on the user responses, they can obtain recommendations regarding appropriate performance evaluation approaches and metrics, including literature references, short video tutorials, and links to available software. Five tasks have been identified for the decision tree: (a) classification, (b) detection/localization, (c) segmentation, (d) time-to-event analysis, and (e) estimation. As an example, the classification branch of the decision tree includes binary and multi-class classification tasks and provides suggestions for methods and metrics as well as software recommendations, and literature references for situations where the algorithm produces either binary or non-binary (e.g., continuous) output and for reference standards with negligible or non-negligible variability and unreliability. The decision tree has been made publicly available on the MIDRC website to assist researchers in conducting task-specific performance evaluations, including classification, detection/localization, segmentation, estimation, and time-to-event tasks.

The Obuchowski-Rockette method has been an important tool for analyzing multi-reader multi-case (MRMC) radiologic imaging data. Although the typical study design for such studies has been the factorial design, where each reader reads each case using each test (modality), sometimes a reader-nested-in-test design is more appropriate. We consider such an example in this talk, where 53 Australian and 15 Singaporean breast radiologists interpreted the same test in their respective locations. In this paper we show how the Obuchowski-Rockette method can be used for analysis of such data, without assuming that the number of readers is the same for each test.

The ideal observer(IO) sets an upper bound for all other classification observers by employing the full statistics of the imaging system and the random object ensemble.¹ This upper bound can be used for the optimization of the imaging system and as a performance measure for other observers. Without binning the data and losing information in the process, the list mode data format has the potential of eliminating null functions² and supporting better performance for clinical tasks. However, it is not easy to track the performance of the ideal observer. A Monte Carlo Markov Chain (MCMC) method³ was proposed for binned data. Here in this work, we present an example of approximating the ideal observer for list mode data given a background known statistically signal known exactly model (BKS/SKE). The receiver operating characteristic (ROC) curve of the IO on list mode data is compared to the corresponding approximation by use of supervised learning methods proposed for binned data,⁴ where convolutional neural networks (CNN)⁵ are used to approximate the probability of the signal present hypothesis given the image, which is a monotone transformation of the IO statistic. The results show the superior performance of IO on list mode data.

One of the main goals in the use of model observers is to serve as an accurate predictor of human observer performance. However, there have been relatively few studies that evaluate model observers in untrained conditions. In this work we evaluate the generalization properties of commonly used model observers using results from a psychophysical study investigating the effect of apodization in low-dose CT from a set of three lesion discrimination tasks. The study involved a total of 24 experimental conditions across three factors (task, system resolution, and apodization). This data allows us to explore the effects of different training regimes on predictive accuracy.
We evaluate the Pre-Whitening Matched Filter (PWMF), “Eye-Filtered” Non-Pre-Whitening (NPWE) and Sparse-Channelized Difference-of-Gaussian (SDOG) models for predictive performance, and we compare various training and testing regimens. These include “training” by using reported values from the literature, training and testing on the same set of experimental conditions, and training and testing on different sets of experimental conditions. Of this latter category, we use both leave-one-condition-out for training and testing as well as a leave-one-factor-out strategy, where all conditions with a given factor level are withheld for testing. Our approach may be considered a fixed-reader approach, since we use all available readers for both training and testing.
Our results show that training models improves predictive accuracy in these tasks, with predictive errors dropping by a factor of two or more in absolute deviation. However, the fitted models are not fully capturing the effects apodization and other factors in these tasks.

Medical imaging systems are commonly assessed and optimized by use of objective measures of image quality (IQ). The performance of the Ideal Observer (IO) acting on imaging measurements has long been advocated as a figure-of-merit to guide the optimization of imaging systems. For computed imaging systems, the performance of the IO acting on imaging measurements also sets an upper bound on task-performance that no image reconstruction method can transcend. As such, estimation of IO performance can provide valuable guidance when designing data-acquisition techniques by enabling the identification of designs that will not permit the reconstruction of diagnostically useful images for a specified task-no matter how advanced the reconstruction method is or plausible the reconstructed images appear. The need for such analyses is urgent because of the ubiquitous development of deep learning-based image reconstruction methods and the fact that they are often not assessed by use of objective image quality measures. However, until recently, estimation of the IO was generally intractable when clinically relevant objects and imaging conditions were assumed. In this work, for the first time, estimates of the IO acting on tomographic imaging measurements were computed with consideration of realistic object variability to establish task-based performance bounds for image reconstruction methods. This was accomplished by use of a recently developed learning-based procedure. Numerical studies that were inspired by breast x-ray computed tomography were conducted to validate and demonstrate the approach. The effectiveness of the approximation method was validated on raw measurements for a signal-known-exactly and background-known-exactly (SKE/BKE) binary signal detection task. A signal-known-statistically and background-known-statistically (SKS/BKS) binary signal detection task was also addressed and the impact of the number of views on IO performance was assessed

Deep convolutional neural networks (CNNs) have demonstrated high accuracy in a wide range of computer vision applications, including medical and biological imaging. Many CNNs are fully supervised learning algorithms, and their performance is directly associated with the quality of the training data labels, which are human-defined. In this work, we investigate the fidelity of human-defined truth for cell detection, segmentation, and classification tasks in multiplex microscopy images. We compare manual annotations from human readers on three tasks. Readers were asked to (1) segment all cells in single-channel fluorescence images of a pannuclear stain (DAPI), (2) segment cells in two-channel fluorescence images (CD20/DAPI), only identifying cells with both nuclear signal (DAPI) and signal from a cell surface marker (CD20), and (3) segment two separate cell classes in three-channel fluorescence images (CD3/CD4/DAPI). In this third task, readers were asked to identify cells that had nuclear signal and were CD3+/CD4- and CD3+/CD4+. By comparing these manual segmentations within and between readers, we demonstrate that human readers show the least variability in single-channel DAPI segmentation (p<<0.05, F test for equal variance). We also compared the agreement of human readers with one another to the agreement of an object-detection network, Yolov5, on cell detection in DAPI images. All pairwise comparisons of human readers with other human readers yielded an average F1-score of 0.83±0.14, and comparisons of Yolov5 with human readers yielded an average F1-score of 0.84±0.12 (p=0.26, Welch’s T test). We therefore demonstrate that out of the provided tasks, DAPI detection provides the highest fidelity ground truth, and were unable to show a difference between Yolov5 and human readers in this task.

Using manually selected cases for breast screener training is time-consuming and prohibitively costly. However, retrieving medical images of breasts via computer vision methods that reflect diagnostically relevant visual features is challenging because the overall appearance variability of whole breasts is high compared to often subtle lesions of interest. Our work aims to develop an automatic and low-costing tool for retrieving and recommending similar full-field digital mammograms (FFDMs) of breast cancer by providing a query image. This tool will help to identify poor reader performance in real-life screening that allows interventions to change practice according to error cases, such as reviewing practice or further training. The core element of this tool is automatic content-based image retrieval. In this paper, we propose an unsupervised method for training an artificial intelligence (AI) model to rank FFDMs according to their similarity in visual features. The proposed method consists of a patch-based pre-processing step, an auto-encoder with Fourier feature mapping and a K-nearest neighbour (K-NN)-based similarity ranking method. To evaluate our method, we used 2500 images (30% abnormal images) from the regular assessment database of UK radiologists over the last ten years. We compared our method with a numerical-data-based method. The results have shown the advantages of our proposed method in retrieving similar cases by considering much richer image information (including but not limited to the breast density, lesion size and location, etc.) without any labour-intensive data labelling.

It is important to evaluate medical imaging artificial intelligence (AI) models for possible implicit discrimination (ability to distinguish between subgroups not related to the specific clinical task of the AI model) and disparate impact (difference in outcome rate between subgroups). We studied potential implicit discrimination and disparate impact of a published deep learning/AI model for the prediction of ICU admission for COVID-19 within 24 hours of imaging. The IRB-approved, HIPAA-compliant dataset contained 8,357 chest radiography exams from February 2020-January 2022 (12% ICU admission within 24 hours) and was separated by patient into training, validation, and test sets (64%, 16%, 20% split). The AI output was evaluated in two demographic categories: sex assigned at birth (subgroups male and female) and self-reported race (subgroups Black/African-American and White). We failed to show statistical evidence that the model could implicitly discriminate between members of subgroups categorized by race based on prediction scores (area under the receiver operating characteristic curve, AUC: median [95% confidence interval, CI]: 0.53 [0.48, 0.57]) but there was some marginal evidence of implicit discrimination between members of subgroups categorized by sex (AUC: 0.54 [0.51, 0.57]). No statistical evidence for disparate impact (DI) was observed between the race subgroups (i.e. the 95% CI of the ratio of the favorable outcome rate between two subgroups included one) for the example operating point of the maximized Youden index but some evidence of disparate impact to the male subgroup based on sex was observed. These results help develop evaluation of implicit discrimination and disparate impact of AI models in the context of decision thresholds

This preliminary study investigates the magnitude of concordance, affecting factors and restrictions when radiologists' make annotations on mammographic images. Annotated data is key to the development of artificial intelligence (AI) tools and errors from annotations can reduce the accuracy of these tool. Two highly experienced radiologists (>20 years’ experience) provided annotations as rectangular regions of interest to mark the location of lesions when they read 856 mammographic images with known cancer signs. Mammographic images were resized to same resolution of 1664 × 768 pixels using bilinear interpolation. We calculated Lin’s concordance correlation coefficient (CCC) between the coordinates in x-axis and y-axis of the 4 corners of the overlapped annotations. The two overlapped annotations in different views (cranio-caudal (CC) and medio-lateral oblique (MLO)) were evaluated for agreement between radiologists. The values of Lin’s CCC were classified in four interpretation levels: the ‘almost perfect’, ‘substantial’, ‘moderate’ and ‘poor’ according to McBride's guide (2015). The results demonstrated ‘almost perfect’, ‘substantial’, ‘moderate’ and ‘poor’ concordance in 50.1%, 29.8%, 9.5% and 10.6% of the total overlapped annotations in the MLO view, with 93.1%, 5.6%, 0.3% and 1.0% of the total overlapped annotations in the CC view, respectively. Overall, the radiologists demonstrated stronger concordance when annotating the CC view compared to the MLO. Breast density (BD) also affected the concordance of the radiologists’ annotations with a decrease in the strength of concordance agreement between breast density classifications, from 0-50% BD = higher concordance to 50-100% BD = lower concordance. Our annotation investigation has implications for AI, where delineation of lesions is often the starting point for training data.

Purpose: Unlike conventional displays, medical image perception in medical extended reality (MXR) applications often involves relative motion between the digital content and the subject with additional sources of noise in the spatial and temporal domains that affects the MXR image quality.
Methods: We describe a spatiotemporal image perception model with static and dynamic signal and noise configurations. The 3D spatiotemporal noise is decomposed into 2D spatial noise and time-dependent noise with motion. The noise in the temporal domain is categorized into time-invariant fixed pattern noise (FPN) and temporal noise that varies per display frame. Visual integration of the moving signal and noise emulates the spatiotemporal image perception of dynamic detection targets in a smooth-pursuit event. A target detection model is implemented to compute the detectability of both low-contrast and high-resolution signal-known-exactly/background-known-exactly (SKE/BKE) targets in various static and dynamic imaging configurations using a non-pre-whitening model observer with eye filter (NPWE).
Results: Smooth pursuit of a moving target suppresses the high-frequency dynamic resolution and noise in the orientation tangential to the motion trajectory. For the dynamic signal and noise configuration, the reduction of both resolution and high-frequency noise results in similar target detectability compared to the reference static image perception. On the other hand, the visibility of a moving target with static FPN is enhanced due to noise aliasing. Visual integration for approximately 33 ms of time-variant temporal noise at 90 Hz display refresh rate reduces the effective noise compared to the FPN by temporal fusion of noise in neighboring display frames.
Conclusion: Spatiotemporal integration of dynamic signal and noise can potentially affect image quality. Complete assessment of image quality in MXR devices needs to consider the contributions from 3D spatiotemporal characteristics.

Quantification of infected lung volume using computed tomography (CT) images can play a critical role in predicting the severity of pulmonary infectious disease. Manual segmentation of infected areas from several CT image slices, however, is not efficient and viable in clinical practice. To assist clinicians in overcoming this challenge, we developed a new method to automatically segment and quantify the percentage of the infected lung volume. First, we used a public dataset of 20 COVID-19 patients, which consists of manually annotated lung and infection masks, to train a new joint deep learning (DL) model for lung and infection segmentation. As for lung segmentation, a Mask-RCNN model was applied to the lung volume with a novel postprocessing technique. Following that, an ensemble model with a customized residual attention UNet model and feature pyramid network (FPN) models was employed for infection segmentation. Next, we assembled another set of 80 CT scans of Covid-19 patients. Two chest radiologists manually evaluated each CT scan and reported the infected lung volume percentage using a customized graphical user interface (GUI). The developed DL-model was also employed to process these CT images. Then, we compared the agreement between the radiologist (manual) and model-based (automated) percentages of diseased regions. Additionally, the GUI was used to let radiologists rate acceptance of the DL-model generated segmentation results. Analyzing the results demonstrate that the agreement between manual and automated segmentation is >95% in 28 testing cases. Furthermore, >53% of testing cases received the top assessment rating scores from two radiologists (between four-five- score). Thus, this study illustrates the feasibility of developing a DL-model based automated tool to effectively provide quantitative evaluation of infected lung regions to assist in improving the efficiency of radiologists in infection diagnosis.

Tools for computer-aided diagnosis based on deep learning have become increasingly important in the medical field. Such tools can be useful, but require effective communication of their decision-making process in order to safely and meaningfully guide clinical decisions. Inherently interpretable models provide an explanation for each decision that matches their internal decision-making process. We present a user interface that incorporates the Interpretable AI Algorithm for Breast Lesions (IAIA-BL) model, which interpretably predicts both mass margin and malignancy for breast lesions. The user interface displays the most relevant aspects of the model’s explanation including the predicted margin value, the AI confidence in the prediction, and the two most highly activated prototypes for each case. In addition, this user interface includes full-field and cropped images of the region of interest, as well as a questionnaire suitable for a reader study. Our preliminary results indicate that the model increases the readers’ confidence and accuracy in their decisions on margin and malignancy.

Purpose: We describe registration accuracy studies of a custom hardware-software system called eeDAP that registers fields of view (FOVs) of a glass slide on a microscope to the digital presentations of regions of interest (ROIs) of whole slide images (WSI) of the same glass slide. In this manuscript, we describe the results of adding new hardware and use the results to size a larger pathologist data collection study.
Methods: We create a registration accuracy task by identifying a visually distinct target. This target will be the center of a WSI ROI and is expected to appear in the center of the microscope FOV. We examined the registration accuracy of 60 ROIs from six slides, alternating registration methods and slide order within each study. We measure the distance between the target and the FOV center (registration error) using an eye piece reticle ruler as the stage moves from target to target. We summarize each error as a success (≤ 5.0 µm) or failure (> 5.0 µm). We completed a multi-reader multicase (MRMC) analysis of the registration successes and failures to estimate the variance components due to the readers and the cases.
Results: When using eeDAP in-focus, accuracy was within 5 µm in more than 97% of the FOVs.
Conclusions: The eeDAP registration methods were robust to new hardware, and the MRMC analysis has provided variance components for sizing future registration accuracy studies to account for the variability from readers and cases.

Artificial intelligence (AI)-based methods are showing substantial promise in segmenting oncologic positron emission tomography (PET) images. For clinical translation of these methods, assessing their performance on clinically relevant tasks is important. However, these methods are typically evaluated using metrics that may not correlate with the task performance. One such widely used metric is the Dice score, a figure of merit that measures the spatial overlap between the estimated segmentation and a reference standard (e.g., manual segmentation). In this work, we investigated whether evaluating AI-based segmentation methods using Dice scores yields a similar interpretation as evaluation on the clinical tasks of quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumor from PET images of patients with non-small cell lung cancer. The investigation was conducted via a retrospective analysis with the ECOG-ACRIN 6668/RTOG 0235 multi-center clinical trial data. Specifically, we evaluated different structures of a commonly used AI-based segmentation method using both Dice scores and the accuracy in quantifying MTV/TLG. Our results show that evaluation using Dice scores can lead to findings that are inconsistent with evaluation using the task-based figure of merit. Thus, our study motivates the need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.

The application of computer-vision algorithms in medical imaging has increased rapidly in recent years. However, algorithm training is challenging due to limited sample sizes, lack of labeled samples, as well as privacy concerns regarding data sharing. To address these issues, we previously developed (Bergen et al. 2022) a synthetic PET dataset for Head & Neck (H&N) cancer using the temporal generative adversarial network (TGAN) architecture and evaluated its performance segmenting lesions and identifying radiomics features in synthesized images. In this work, a two-alternative forced-choice (2AFC) observer study was performed to quantitatively evaluate the ability of human observers to distinguish between real and synthesized oncological PET images. In the study eight trained readers, including two board-certified nuclear medicine physicians, read 170 real/synthetic image pairs presented as 2D-transaxial using a dedicated web app. For each image pair, the observer was asked to identify the “real” image and input their confidence level with a 5-point Likert scale. P-values were computed using the binomial test and Wilcoxon signed-rank test. A heat map was used to compare the response accuracy distribution for the signed-rank test. Response accuracy for all observers ranged from 36.2% [27.9-44.4] to 63.1% [54.8-71.3]. Six out of eight observers did not identify the real image with statistical significance, indicating that the synthetic dataset was reasonably representative of oncological PET images. Overall, this study adds validity to the realism of our simulated H&N cancer dataset, which may be implemented in the future to train AI algorithms while favoring patient confidentiality and privacy protection.

Dose reduction remains an important goal in interventional x-ray. We propose an image quality (IQ) measure called the visibility overshoot index. Given a patient image and a specified clinical task, the index quantifies the maximum acceptable dose reduction. The dose control system can then use this information to deliver the minimum dose necessary for detection of clinical signals, reducing unnecessary radiation exposure. We developed an experimental visual model to estimate signal detectability as a function of image features such as noise and signal contrast. The model is used to find a feature’s threshold–the maximum change in noise or signal contrast where signal detectability remains possible. An automated algorithm measures the magnitudes of these features on a frame. Visibility overshoot is expressed in terms of the image features: the noise overshoot and contrast overshoot indices are the ratio of the threshold to measured noise/contrast. The indices demonstrate good agreement with detector dose, channelized hotelling observer results, and clinicians’ judgments. In our study of a cylindrical object phantom acquired at seven dose levels, we found that the noise overshoot index is linearly related to the square root of detector dose and the CHO detectability index, with Pearson correlation 0.995–1.0 for signals 1-4 mm diameter. For interventional cardiology and neurology sequences acquired at standard and 25–30% dose, the index and clinicians rank IQ similarly. Results on the phantom suggest at least 15% dose reduction could be achieved in fluoroscopy mode. Our patient-specific IQ approach could bring additional dose savings to clinical practice.

Undersampling in the frequency domain (k-space) in MRI accelerates the data acquisition. Typically, a fraction of the low frequencies is fully collected and the rest are equally undersampled. We used a fixed 1D undersampling factor of 5x where 20% of the k-space lines are collected but varied the fraction of the low k-space frequencies that are fully sampled. We used a range of fully acquired low k-space frequencies from 0% where the primary artifact is aliasing to 20% where the primary artifact is blurring in the undersampling direction. Small lesions were placed in the coil k-space data for fluid-attenuated inversion recovery (FLAIR) brain images from the fastMRI database. The images were reconstructed using a multi-coil SENSE reconstruction with no regularization. We conducted a human observer two-alternative forced choice (2-AFC) study with a signal known exactly and a search task with variable backgrounds for each of the acquisitions. We found that for the 2-AFC task, the average human observer did better with more of the low frequencies being fully sampled. For the search task, we found that after an initial improvement from having none of the low frequencies fully sampled to just 2.5%, the performance remained fairly constant. We found that the performance in the two tasks had a different relationship to the acquired data. We also found that the search task was more consistent with common practice in MRI where a range of frequencies between 5% and 10% of the low frequencies are fully sampled.

Model observers are one of the most common tools to evaluate signal detectability on medical images. There is a large literature of model observers applied to simple signals such as disks or Gaussian blobs. However, these signals do not represent realistic scenarios where lesion shapes are irregular and lesion location is unknown. For instance, in breast imaging, masses can be elongated, spiculated or bumpy. We study how different model observers perform on irregularly shaped signals including the Non-Pre-Whitening (NPW) and two variations of the Channelized Hotelling Observer (CHO), one with symmetrical Laguerre-Gauss channels and one with adaptive Laguerre-Gauss channels. The novel adaptive Laguerre-Gauss channels are built taking into account the shape of the lesion by modifying the distance matrix used to build channels that adopt the shape of the signal. We embedded four different signal shapes in 2D realistic backgrounds in a digital mammography image generated using the VICTRE in silico breast imaging system and studied the models’ performance for detection and search. Our results show that the adaptive channels perform better than symmetrical channels with the most irregular signal (a star and an elongated mass).

The Noise Power Spectra (NPS) only characterizes first and second order statistics associated with noise in Computed Tomography (CT) reconstructions. The purpose of this work is to characterize the impact of the higher order statistics on perception of noise texture for a variety of reconstruction algorithms. Images of a 32 cm water phantom were acquired on the Aquilion ONE Genesis CT system and reconstructed with AiCE deep learning reconstruction (DLR), model-based iterative reconstruction (MBIR), hybrid iterative reconstruction (AIDR), and filtered backprojection (FBP). Regions of interest (ROIs) of 100x100pixels were extracted from the center of the images and 4^th order statistics of each ROI were assessed via excess kurtosis measurement. Pure Gaussian noise counterpart image datasets with the same mean, standard deviation (SD), and NPS as each acquired data condition were also generated by convolving random white noise with the root-NPS of the acquired data. Nine naïve observers were tasked with distinguishing the acquired noise image from the pure Gaussian counterpart via a two-alternative forced choice experiment. Excess kurtosis in the image ROIs was 0.01 for FBP, 0.74 to 0.85 for FIRST, 0.03 to 0.08 for AIDR, and -0.13 to 0.21 for AiCE. Results showed the FBP images appeared indistinguishable from their pure Gaussian counterparts with a Percent Correct (PC)=54%, while MBIR images were readily distinguishable from their pure Gaussian counterparts, PC=98 to 100%. DLR and AIDR images are more difficult to distinguish from their pure Gaussian counterparts, with the PC ranging from 58% to 88%. The discriminability index derived from the PCs correlated strongly with excess kurtosis.

Background parenchymal enhancement (BPE) is an independent risk factor for breast cancer that is subjectively defined by the relative volume and intensity of enhancement in normal fibroglandular breast tissue after contrast injection in dynamic contrast-enhanced (DCE) MRI. Given the relative increasing use of 3T instead of 1.5T MRI, this study evaluates the influence of magnet strength on radiologist BPE ratings and on performance of our previously-developed computer method for objective BPE scoring. The retrospectively-collected dataset consisted of 661 conventional breast DCE-MR exams (394 1.5T, 267 3T) from 350 patients with diagnosed lesions (2007-2015) and 311 high-risk screening patients (2009-2022). For the subset of 350 cases, receiver operating characteristic analysis was performed to determine the predictive value of the computed BPE scores relative to radiologists’ BPE ratings. Resulting computer BPE scores showed statistically significant correlations with radiologist ratings and performed statistically superior to guessing in the classification tasks. For the entire dataset, the prevalence of BPE ratings was explored for each magnet strength. We found that radiologists perceived higher BPE ratings on 3T than on 1.5T images, however we failed to show a statistical difference between 1.5T and 3T distributions of the computer BPE scores. We speculate that the transition from 1.5T to 3T MRI has influenced radiologist perception of BPE and can influence the performance of machine-learning methods for BPE quantification.

Display technology is rapidly improving with increasingly brighter displays. Since the contrast perception of the human visual system improves with increasing luminance, low luminance areas in medical images should be avoided. Older displays were less bright and had a high surface reflection, so the only possible way to increase contrast was to lower the black level by reducing the room illuminance. A dark viewing room soon became a prerequisite when viewing medical images, and although not ideal, it was necessary due to technological limitations. However, brighter displays offer new possibilities. This study determined the contrast thresholds in the dark, mid-gray and bright regions of an image displayed using three different luminance ranges: 1-250, 6-500 and 12-750 cd/m². All three luminance ranges covered the same number of just noticeable differences (JNDs) but had vastly different luminance ratios. The visibility of two bar-pattern frequencies were studied: four and 16 cycles per degree. No differences in contrast threshold could be found between the three luminance ranges, indicating that high black levels and low luminance ratios are possible to use without degrading image contrast, if the display luminance range covers enough JNDs. The high frequency patterns required ten times the contrast of the low frequency patterns to be visible. Contrast stability was also studied, and the results indicate that it is easier to maintain stable image contrast if the display minimum luminance is high. The amount of reflected luminance is irrelevant for stability if the room illumination is stable.

The purpose of this study is to compare interpretation efficiency of radiologists reading radiographs on 6 megapixel (MP) vs 12 MP monitors. Our method compares two sets of monitors in two phases: in phase I, radiologists interpreted using a 6MP, 30.4 inch (Barco Coronis Fusion) and in phase II, a 12MP, 30.9-inch (Barco Nio Fusion) is used. Nine chest and three musculoskeletal radiologists each batch-interpreted an average of 115 radiographs in phase I and 115 radiographs in phase II as part of routine clinical work. Radiologists were blinded to monitor resolution. Interpretation times per radiograph was noted from dictation logs. We found that interpretation was faster on 12 MP monitor by 6.88 (p = 0.002) and 6.76 seconds (8.7%) (p < 0.001) for chest radiographs only and combined chest and musculoskeletal radiographs, respectively. When evaluating musculoskeletal radiographs alone, the improvement in reading times with 12 MP monitor was 6.76 seconds (p = 0.111). Interpretation of radiographs on 12 MP monitors was ~8.7% faster than on 6 MP monitors.

Computer-aided diagnostic schemes have been developed with the primary aim of supporting diagnostic accuracy in the investigation of mammography images. A robust mammograms database is a priority requirement to assist in testing the effectiveness of techniques associated with these schemes. However, such datasets–with information on radiological and biopsy reports, different types of findings and good quality images-are difficult to be available, mainly due to restrictions of different radiology centers and hospitals or limited to the research team involved. Aiming to overcome these issues, we are developing an alternative based on images of a structured breast phantom, previously developed in our group. Although having been validated in terms of their physical characteristics, the investigation on the images produced from this phantom exposure on a digital mammography system is an important task with regard to their appearance compared to actual breasts images. For this purpose, a software was developed for managing comparative tests between images from actual breasts and from this phantom, considering internal regions of interest. The intention is to evaluate how much the simulated and real images are confused according to the human visual perception of different observers, seeking to validate the use of this breast phantom for structuring a new mammography images database aimed at evaluation tests of CAD schemes in mammography. Results were promising showing a variation of 50-60% in the rate of correct answers among observers, indicating a reasonable level of confusion between the two types of images.

Skeletal maturity assessment is an important step in the treatment of adolescent idiopathic scoliosis (AIS). Traditional methods rely on the bone age assessment using 2D X-ray radiography. Assessment is generally performed using the Risser stages that is routinely used to assess the skeletal maturity through the observation of the level of ossification of iliac crests. This bone maturity assessment method is preferred in AIS but in practice, shows a rather high-level interobserver variability. This study aims to use an automatic Risser stage classification for a longitudinal study of follow-up visits to observe growth indicators of AIS patients. A regression model will then be used to evaluate the maturity changes of patients from à Risser stage to another. For the classification task, the pre-trained model of VGG16 was implemented with Python 3.10. The network parameters were changed since the task we were training it for contained a smaller dataset. The first experiments of this work were for the classification of the patient’s Risser signs. After several tests of optimization of the SVR classifiers hyperparameter, a mean square error of 0.38, a mean absolute error of 0.31 and an R² of 0.33. An optimization of the network and a pre-processing of the images will be done in the next phases of this project.

Deep learning techniques to segment Wilms tumor typically use a single MRI sequence as input. The aim of this study was to assess whether multiparametric MRI input improves Wilms tumor segmentation. 45 patients were consecutively included, of which 36 were used for training and nine for testing. All seven input combinations of postcontrast T₁-weighted imaging, T₂-weighted imaging, and diffusion weighted imaging (DWI) were used for nnU-Net training. Dice scores and the 95th percentile of the Haussdorf distance (HD95) were used to evaluate the input combinations. The median Dice score was highest when combining all MRI sequences (Dice = 0.93), the median HD95 was lowest when combining postcontrast T₁-weighted imaging and DWI (HD95 = 5.4 mm). Single-parametric DWI input performed significantly worse than other input combinations (median Dice = 0.64, median HD95 = 29.5 mm, p = 0.004). All other combinations, including standalone sequences, showed similar performance to each other. Our results suggest that adding sequences to standalone T₁-weighted or T₂-weighted imaging does not significantly improve segmentation results.

Cerebral vascular abnormalities, such as unruptured intracranial aneurysms (UIAs), can result in poor clinical outcomes or death. A statistical atlas of the cerebral vasculature based on healthy controls can be used to identify abnormalities in patients. This atlas was registered to 19 patients with an UIA and 18 healthy controls. Z-score values were computed for the vessels in the Circle of Willis, indicating if and where the vessel radius deviates from the atlas. A color-coded Z-score map was made, to aid radiologists in identifying cerebral vascular abnormalities. Results showed that patients with UIAs have statistically significantly higher Z-score values than healthy controls. In addition, in 17/19 patients the Z-score values at the location of the UIA were statistically significantly higher than elsewhere in the Circle of Willis. This indicates that this technique can both be used to identify patients-at-risk with an abnormal cerebral vasculature, as well as guide visual search for UIAs with 89 % detection sensitivity.

In digital mammography, the physics of the acquisition system and post-processing algorithms can cause image noise to be spatially correlated. Noise correlation is characterized by non-constant noise power spectral density and can negatively affect image quality. Although the literature explores ways to quantify the frequency dependence of noise in digital mammography, there is still a lack of studies that explore the effect of this phenomenon on clinical tasks. Thus, the aim of this work is to evaluate the impact of noise correlation on the quality of digital mammography and the detectability of lesions using a virtual clinical trial (VCT) tool. Considering the radiographic factors of a standard full-dose acquisition, VCT was used to generate two sets of images: one containing mammograms corrupted with correlated noise and the other with uncorrelated (white) noise. Clusters of five to seven microcalcifications of different sizes and shapes were computationally inserted into the images at regions of dense tissue. We then designed a human observer study to investigate performance on a clinical task of locating microcalcifications on digital mammography from both image sets. In addition, nine objective image quality metrics were calculated on mammograms. The results obtained with four medical physicists showed that the average performance in localization was 72% for images with correlated noise and 95% with uncorrelated noise. Thus, our results suggest that correlated noise promotes a greater reduction in the conspicuity of subtle microcalcifications than uncorrelated noise. Furthermore, only four of the nine objective quality metrics calculated in this work were consistent with the results of the human observer study, highlighting the importance of using appropriate metrics to assess the quality of corrupted images with correlated noise. The source code for our framework is publicly available at https://github.com/LAVI-USP/SPIE2023-ImageQuality.

Introduction: Breast cancer is the most common cancer among women in China and early detection is key to reducing mortality. This study aimed to understand diagnostic performances of Chinese radiologists between FFDM (full-field digital mammography) and DBT (digital breast tomosynthesis) images in terms of lesion features and reader characteristics.
Methods: 32 Chinese radiologists read two mammogram test sets to identify cancer cases and to detect lesions. The first set was of FFDM images (60 cases, 21 cancers) and the second was of DBT images (35 cases, 15 cancers). The accuracy in cancer case detection and lesion detection of radiologists in each test set were analysed. Comparison of diagnostic performances of radiologists with different working experiences were also undertaken. Results were compared using the Wilcoxon Sign Rank and Mann-Whitney U tests.
Results: Chinese radiologists recorded higher diagnostic accuracy with FFDM than DBT for detecting certain lesion types (calcifications, architectural distortion, mixed types) and lesions ≤ 10 mm. There was no significant difference in the accuracy for cancer case detection between FFDM and DBT. Radiologists who had more than eight years working experience, read more than 60 cases per week or had no DBT training had significantly higher lesion accuracy with FFDM than DBT.
Conclusion: Chinese radiologists had higher lesion accuracy with FFDM in certain lesion types and sizes than DBT. This may be related to the lack of appropriate DBT training for radiologists in China.

It has been suggested that medical imaging systems should be evaluated and optimized by use of task-based measures of image quality (IQ). Task-based measures of IQ summarize the performance of an observer at a specific task (e.g., tumor detection). The Hotelling observer (HO) is a commonly employed numerical observer for evaluating and optimizing medical imaging systems. However, the computation of the HO can be intractable when huge covariance matrices of the image data need to be inverted. One way to address this issue is to apply a set of channels to the image data and subsequently compute the HO on the channelized data. When the channels are efficient, the HO performance can be approximated by the performance of the channelized Hotelling observer (CHO). However, it remains unclear how efficient channels can be learned and subsequently employed when performing image processing tasks. In this work, I propose a task-aware method for training denoising autoencoders (DAEs) for establishing efficient channels that can be employed for image denoising. It is demonstrated that the HO performance can be closely approximated by use of the proposed task-aware DAE-learned channels. In addition, the images produced by the proposed task-aware DAEs can achieve improved signal detectability evaluated by a foveated CHO, which was developed for modeling human visual systems.

Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several applications in medical imaging that include unconditional medical image synthesis, image translation, and optimization of imaging systems. However, the extent to which a GAN learns image statistics that are relevant to a diagnostic task is unknown. In this work, canonical stochastic image models (SIMs) that simulate realistic mammographic textures are employed to evaluate GAN-based SIMs with respect to detection, detection-localization, and detection-estimation tasks. It is shown that the specific GAN architecture considered has higher propensity to generate statistics that confound the observers performing the three considered tasks. This work highlights the need for continued development of objective metrics for evaluating GANs.

Previous research has revealed that Vietnamese radiologists had lower diagnostic efficacy in interpreting mammograms than radiologists from Western countries. This study investigated the improvement in diagnostic performances of Vietnamese doctors in breast cancer detection via VIETRAD (VIEtnam: Transformation of Radiological Detection) program. Data of 33 participants who completed three training sessions containing normal and cancer mammographic cases from Australia and Vietnam were assessed in sensitivity, specificity, ROC and JAFROC. Results show that Vietnamese doctors have improved their diagnostic accuracy in identifying normal and cancer cases on mammograms across different levels of breast density.

Deep-learning (DL)-based methods have shown significant promise in denoising myocardial perfusion SPECT images acquired at low dose. For clinical application of these methods, evaluation on clinical tasks is crucial. Typically, these methods are designed to minimize some fidelity-based criterion between the predicted denoised image and some reference normal-dose image. However, while promising, studies have shown that these methods may have limited impact on the performance of clinical tasks in SPECT. To address this issue, we use concepts from the literature on model observers and our understanding of the human visual system to propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. The proposed method was objectively evaluated on the task of detecting perfusion defect in myocardial perfusion SPECT images using a retrospective study with anonymized clinical data. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images. The results show that by preserving task-specific information, DL may provide a mechanism to improve observer performance in low-dose myocardial perfusion SPECT.

Deep learning algorithms for detection and segmentation have been shown to be vulnerable to single-pixel attacks. These attacks can lead to catastrophic failure of the deep learning algorithm. In the case of biomedical imaging, this can result in significant damage to clinical outcomes. While single-pixel attacks have been studied within the field of digital pathology, they have yet to be studied within the realm of radiology, in particular with volumetric U-Net or V-Net architectures. In this work, we demonstrated that using gradcam++, we could identify vulnerable voxels for the single-voxel attacks that were slightly negative in value towards the boundary of kidney segmentations that lead to the significant distortion of the output kidney classification. Figure 1 demonstrates the graphical abstract for this work.

Recently published guidelines recommend the validation of models for medical image processing against variations of image acquisition parameters. This work presents a novel concept for systematic tests that investigate a model’s performance to MRI acquisition shifts in brain scans of multiple sclerosis patients. Contrast changes of T2-weighted FLAIR images related to changes of two scan parameters (TE, TI) are simulated using artificial data. These images were created from 114 brain MRI, normal tissue segmentation masks and expert MS lesion masks (NAMIC, OpenMS, MSSEG2, ISBI15, LC08). Contrast changes were simulated by using the FLAIR signal equation. The experiments evaluate the F1 scores of models trained on images based on different uniform and non-uniform acquisition parameters when tested on typical acquisition parameter shifts. The acquisition parameter variations of the experiments are based on guidelines for quality assurance and the analysis of protocols and images from different institutions and open datasets. The dependence of the F1 score on both parameters could be approximated by linear functions with R2 scores of 0.94 to 0.98. Thus, linear functions can be used to model the dependence of the F1 score on the influencing factors, thereby allowing for the derivation of a "save" range of image acquisition parameters to meet a desired performance metric. The segmentation model was more sensitive to changes in TE compared to TI. The maximum lesion F1 loss when applying the models to out-of-distribution data ranged from 0.04 to 0.36 and was significant in all the experiments, even when the model was trained on data representing scans of different contrast (p<0.01). This underlines the need for testing segmentation models against acquisition shifts.

Purpose: Variability in observer performance BICR is common but not well understood and various measures like AR, AAR, RDI help quantify it which leads to multiple complex data points. Network analysis uses mathematically based algorithms to characterize the components of a network of entities and identifying, visualizing, and analysing their relationships. In a network, variables are represented by nodes, the relationships represented by edges between these nodes. The visualization technique involves mapping relationships among entities based on the symmetry or asymmetry of data. Maps from data points generated during double read adjudication study can provide the performance of each reader pair primarily based on AAR.
Methods: Adjudication data from four oncology clinical trials with 2163 subjects, 16937 post-baseline responses was analyzed. Performance metrics included number of cases, adjudication rate, adjudication agreement rate for each read and reader pair. The data were aggregated and prepared for network analysis in Python-a high-level, cross-platform, and open-sourced programming language released under a GPL-compatible license. Python Software Foundation (PSF), a non-profit organization, holds the copyright. Url-https://www.python.org Version 3.9.0
Results: This graphic visualization provides simplistic organization of a complicated data analysis and supports the quality monitoring process of independent reviews. The tool provides a snapshot of the review performance of all the readers in the trial allowing the study team to investigate and intervene in a timely manner with the intent of supporting robust and accurate data analysis.
Conclusions: Network analysis plots for reader performance metrics in BICR provide excellent visual mapping to interpret multiple critical metrics in a single plot which would otherwise require multiple plots and tables. Timely review of these plots during the trial can help demonstrate the effectiveness of interventions as well.

Multiple imaging modalities are commonly jointly used for investigating biological phenomena or diagnosis purposes. In this study, we propose a deep-learning-based cross-modality imaging technique that utilizes one imaging modality to computationally predict another. A novel neural network architecture, featuring recurrent multi-stage refinement controlled by gated activation, was developed for this purpose. To demonstrate the effectiveness of the proposed method, we conducted experiments on predicting organelle fluorescence images from stimulated Raman scattering (SRS) imaging. The results of the experiments indicate that our method outperforms the current state-of-the-art techniques across multiple datasets, in terms of both accuracy and efficiency. The neural network architecture was able to produce high-quality predictions with clear boundaries and high prediction accuracy through the multi-stage refinement process. The proposed method presents a versatile framework that addresses the limitations of current deep-learning-enabled cross-modality image prediction techniques and has potential applications in the field of medical and biological imaging.

Purpose: Blinded independent central review is recommended by the US FDA for registration of oncology trials as it provides bias-free image assessment and avoidance of potential unblinding of patient data. Double read with adjudication is a highly advocated review model used in such trials. Disagreement between readers is natural and inevitable. Radiological disagreement rates or Adjudication Rate (AR) of 30–65% are reported by several papers since 1959 for different oncologic indications. The aim of the study is to develop and use an algorithm to identify reader pair with predicted high AR and investigate if overall study AR can be kept constant or improved by assigning less cases to a reader pair with high AR.
Methods: A retrospective analysis was performed of 285 subjects with 3351 post-baseline timepoints reviewed by board-certified radiologist reviewers using Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria in a BICR set up. The reader adjudication rate was calculated and analyzed throughout the duration of review. The distribution of cases per reader and distribution of cases per reader pair was calculated and overall study AR, and each reader pair AR were calculated.
Data Analysis Methods: Data was prepared and analyzed using linear regression with MS Excel and R programming script (R version 4.1.2 (2021-11-01) -- "Bird Hippie" augmented by RStudio 2022.07.0+548 "Spotted Wakerobin".
Results: Using the data at completion of 50% reads, predicted AR per reader pair was found to correlate well for five out of six reader pairs on the study. This predicted AR can then be used to assign more cases to reader pair with low AR and less cases to reader pair with high AR to keep study level variability low.
Conclusions: Predictive modeling of AR using linear regression can provide an insight into variability of each reader pair, which in turn determines the AR of that reader pair and collectively determines study AR. However, it is still not clear to what level of prioritization in case assignment to specific pairs can be considered acceptable and not artificial.

This paper, originally published on 3 April 2023, was replaced with a corrected/revised version on 7 July 2023. If you downloaded the original PDF but are unable to access the revision, please contact SPIE Digital Library Customer Service for assistance.