A preliminary comparison of different methods for observer performance estimation

Francesc Massanes; Jovan G. Brankov

doi:10.1117/12.2007229

28 March 2013 A preliminary comparison of different methods for observer performance estimation

Francesc Massanes, Jovan G. Brankov

Proceedings Volume 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment; 86730Z (2013) https://doi.org/10.1117/12.2007229
Event: SPIE Medical Imaging, 2013, Lake Buena Vista (Orlando Area), Florida, United States

Abstract

In medical imaging, image quality is assessed by the degree to which a human observer can correctly perform a given diagnostic task. Therefore the image quality is typically quantified by using performance measurements from decision/detection theory like the receiver operation characteristic (ROC) curve and the area under ROC curve (AUC). In this paper we compare five different AUC estimation techniques, widely used in the literature, including parametric and non-parametric methods. We compared each method by equivalence hypothesis testing using a model observer as well as data sets from a previously published human observer study. The main conclusions of this work are 1) if a small number of images are scored, one cannot tell apart different AUC estimation methods due to large variability in AUC estimates, regardless whether image scores are reported on a continuous or quantized scale. 2) If the number of scored images is large and image scores are reported on a continuous scale, all tested AUC estimation methods are statistically equivalent.

Citation Download Citation

Francesc Massanes and Jovan G. Brankov "A preliminary comparison of different methods for observer performance estimation", Proc. SPIE 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, 86730Z (28 March 2013); https://doi.org/10.1117/12.2007229

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available