Inter-reader variability in alternate forced choice studies

W. Huda; K. M. Ogden; E. Samei; E. M. Scalzetti; R. L. Lavallee; M. L. Roskopf

doi:10.1117/12.770618

Translator Disclaimer

6 March 2008 Inter-reader variability in alternate forced choice studies

W. Huda, K. M. Ogden, E. Samei, E. M. Scalzetti, R. L. Lavallee, M. L. Roskopf

Author Affiliations +

Proceedings Volume 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment; 691711 (2008) https://doi.org/10.1117/12.770618
Event: Medical Imaging, 2008, San Diego, California, United States

Abstract

In this study, we investigated differences in detection performance for twelve observers who each generated a CT contrast detail curve. An anthropomorphic newborn phantom's abdomen was imaged using a GE Light Speed CT scanner (4-slice). Alternate Forced Choice (AFC) experiments were performed with lesions sizes ranging from 2.5 to 12.5 mm to determine the intensity needed to achieve 92% correct (I_92%). Following training, twelve readers consisting of (2 technologists, 4 college students, 4 medical students, and 2 radiology residents) generated a single contrast detail curve. Eight readers produced approximately linear contrast detail curves while the remaining four readers required a second order polynomial fit because of reduced performance when detecting the largest (i.e., 12.5 mm) lesion. For the three smallest lesions, the coefficient of variation between the twelve readers was ~12%, which increases with increasing lesion size to ~23% for 12.5 mm lesion size. The ratio of the maximum I_92% to minimum I_92% values was ~1.6 for the smallest lesions, which increased to a factor of ~2.1 for the 12.5 mm lesion. Our results show that minimizing inter-reader variability in our AFC experiments could be achieved by eliminating the largest lesion that cause detection problems in one third of observers. The combined experimental data showed that the slope of the contrast detail curve was -0.42, lower than the value of -1.0 predicted by the Rose model, suggesting that the noise texture in CT associated with both quantum mottle and anatomic structure is an important factor affecting detection of these lesions.

Citation Download Citation

W. Huda, K. M. Ogden, E. Samei, E. M. Scalzetti, R. L. Lavallee, and M. L. Roskopf "Inter-reader variability in alternate forced choice studies", Proc. SPIE 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment, 691711 (6 March 2008); https://doi.org/10.1117/12.770618

ACCESS THE FULL ARTICLE