Analysis of double reading in an observer study

Yulei Jiang

doi:10.1117/12.814945

Translator Disclaimer

12 March 2009 Analysis of double reading in an observer study

Yulei Jiang

Author Affiliations +

Proceedings Volume 7263, Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment; 726316 (2009) https://doi.org/10.1117/12.814945
Event: SPIE Medical Imaging, 2009, Lake Buena Vista (Orlando Area), Florida, United States

Abstract

Previously we showed based on theoretical analysis that it is possible to attain greater diagnostic performance from appropriately combining the diagnostic opinions of two or more equally skilled readers. Such gain in performance is available from combining the readers' "latent decision variables" that are accessible through ROC analysis, but is generally ambiguous at best if the readers' binary decisions with regard to clinical actions (e.g., recall vs. annual screening mammogram) are combined. We now analyze the data of an observer study. In this observer study, ten radiologists interpreted 104 cases of mammograms containing clustered microcalcifications in a diagnostic-study setting to decide whether to recommend biopsy. They also reported diagnostic confidence on a quasi-continuous scale that the calcifications indicated malignancy. A previous analysis showed that combining the radiologists' binary decisions (biopsy vs. no biopsy) would change both sensitivity and specificity generally along the radiologists' single-reading, average, ROC curve but would not increase the diagnostic performance. Combining two radiologists' "latent decision variables" resulted in small increases in the ROC curves consistent with the theoretical predictions. However, the shapes of the single-reading ROC curves were inconsistent with the expectation of the clinical diagnostic-study setting because all benign cases in the observer study were difficult-to-diagnose cases (all cases clinically biopsied). The double-reading results would have been different, and gains in diagnostic performance possible, if the ROC curve shape more accurately resembled that of clinical practice. There is need to estimate the ROC curve of clinical practice.

Citation Download Citation

Yulei Jiang "Analysis of double reading in an observer study", Proc. SPIE 7263, Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment, 726316 (12 March 2009); https://doi.org/10.1117/12.814945

ACCESS THE FULL ARTICLE