Pooling MRMC forced-choice data

Brandon D. Gallas; Gene A. Pennello

doi:10.1117/12.709628

30 March 2007 Pooling MRMC forced-choice data

Brandon D. Gallas, Gene A. Pennello

Author Affiliations +

Proceedings Volume 6515, Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment; 651506 (2007) https://doi.org/10.1117/12.709628
Event: Medical Imaging, 2007, San Diego, CA, United States

Abstract

There are at least two sources of variance when estimating the performance of an imaging device: the doctors (readers) and the patients (cases). These sources of variability generate variances and covariances in the observer study data that can be addressed with multi-reader, multi-case (MRMC) variance analysis. Frequently, a fully-crossed study design is used to collect the data; every reader reads every case. For imaging devices used during in vivo procedures, however, a fully-crossed design is infeasible. Instead, each patient is diagnosed by only one doctor, a doctor-patient study design. Here we investigate percent correct (PC) under this doctor-patient study design. From a probabilistic foundation, we present the bias and variance of two statistics: pooled PC and reader-averaged PC. We also present variance estimates of these statistics and compare them to naive estimates. Finally, we run simulations to assess the statistics and the variance estimates. The two PC statistics have the same means but different variances. The variances depend on how patients are distributed among the readers and the amount of reader variability. Regarding the variance estimates, the MRMC estimates are unbiased, whereas the naive estimates bracket the true variance and can be extremely biased.

Citation Download Citation

Brandon D. Gallas and Gene A. Pennello "Pooling MRMC forced-choice data", Proc. SPIE 6515, Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment, 651506 (30 March 2007); https://doi.org/10.1117/12.709628

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Error analysis

Monte Carlo methods

Statistical analysis

Binary data

Computer simulations

Imaging devices

In vivo imaging

Show All Keywords

Keywords/Phrases

Search In:

Publication Years