Inter-reader variability in alternate forced choice studies

W. Huda; K. M. Ogden; E. Samei; E. M. Scalzetti; R. L. Lavallee; M. L. Roskopf

doi:10.1117/12.770618

6 March 2008 Inter-reader variability in alternate forced choice studies

W. Huda, K. M. Ogden, E. Samei, E. M. Scalzetti, R. L. Lavallee, M. L. Roskopf

Proceedings Volume 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment; 691711 (2008) https://doi.org/10.1117/12.770618
Event: Medical Imaging, 2008, San Diego, California, United States

Abstract

In this study, we investigated differences in detection performance for twelve observers who each generated a CT contrast detail curve. An anthropomorphic newborn phantom's abdomen was imaged using a GE Light Speed CT scanner (4-slice). Alternate Forced Choice (AFC) experiments were performed with lesions sizes ranging from 2.5 to 12.5 mm to determine the intensity needed to achieve 92% correct (I_92%). Following training, twelve readers consisting of (2 technologists, 4 college students, 4 medical students, and 2 radiology residents) generated a single contrast detail curve. Eight readers produced approximately linear contrast detail curves while the remaining four readers required a second order polynomial fit because of reduced performance when detecting the largest (i.e., 12.5 mm) lesion. For the three smallest lesions, the coefficient of variation between the twelve readers was ~12%, which increases with increasing lesion size to ~23% for 12.5 mm lesion size. The ratio of the maximum I_92% to minimum I_92% values was ~1.6 for the smallest lesions, which increased to a factor of ~2.1 for the 12.5 mm lesion. Our results show that minimizing inter-reader variability in our AFC experiments could be achieved by eliminating the largest lesion that cause detection problems in one third of observers. The combined experimental data showed that the slope of the contrast detail curve was -0.42, lower than the value of -1.0 predicted by the Rose model, suggesting that the noise texture in CT associated with both quantum mottle and anatomic structure is an important factor affecting detection of these lesions.

Citation Download Citation

W. Huda, K. M. Ogden, E. Samei, E. M. Scalzetti, R. L. Lavallee, and M. L. Roskopf "Inter-reader variability in alternate forced choice studies", Proc. SPIE 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment, 691711 (6 March 2008); https://doi.org/10.1117/12.770618

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available