Optimal use of rating scales in ROC analysis

R. M. Dawood; Andrew Todd-Pokropek; J. O.M.C. Craig; J. H. Highman; A. W. Porter

doi:10.1117/12.19029

1 August 1990 Optimal use of rating scales in ROC analysis

R. M. Dawood, Andrew Todd-Pokropek, J. O.M.C. Craig, J. H. Highman, A. W. Porter

Proceedings Volume 1234, Medical Imaging IV: PACS Systems Design and Evaluation; (1990) https://doi.org/10.1117/12.19029
Event: Medical Imaging '90, 1990, Newport Beach, CA, United States

Abstract

Observers participating in ROC studies are usually required to estimate the confidence with which each observation is made. With a discrete scale, the rating, or score, normally falls into one of 5categories, ranging from 'definitely normal' to 'definitely abnormal'. However, a major problem in data analysis from ROC studies has been found to be caused by observers who have not used the rating scale in a uniform manner, and have made many responses corresponding to the two extreme categories with few responses falling in the middle. The use of a continuous rating scale, with a point selected using a mouse, has assisted in analysis, but only to a limited extent. It has therefore been suggested elsewhere that it is desirable to force observers to select intermediate points. The effect of such an approach on ROC curves was studied by asking a group of observers to re-score a set of difficult clinical images, after training and with continuous feedback on their compliance. Although the resulting fall in the ROC curves was not statistically significant, it is considered unwise to force observers to report in what to them appears to be an unnatural manner.

Citation Download Citation

R. M. Dawood, Andrew Todd-Pokropek, J. O.M.C. Craig, J. H. Highman, and A. W. Porter "Optimal use of rating scales in ROC analysis", Proc. SPIE 1234, Medical Imaging IV: PACS Systems Design and Evaluation, (1 August 1990); https://doi.org/10.1117/12.19029

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available