Translator Disclaimer
30 March 2007 Pooling MRMC forced-choice data
Author Affiliations +
There are at least two sources of variance when estimating the performance of an imaging device: the doctors (readers) and the patients (cases). These sources of variability generate variances and covariances in the observer study data that can be addressed with multi-reader, multi-case (MRMC) variance analysis. Frequently, a fully-crossed study design is used to collect the data; every reader reads every case. For imaging devices used during in vivo procedures, however, a fully-crossed design is infeasible. Instead, each patient is diagnosed by only one doctor, a doctor-patient study design. Here we investigate percent correct (PC) under this doctor-patient study design. From a probabilistic foundation, we present the bias and variance of two statistics: pooled PC and reader-averaged PC. We also present variance estimates of these statistics and compare them to naive estimates. Finally, we run simulations to assess the statistics and the variance estimates. The two PC statistics have the same means but different variances. The variances depend on how patients are distributed among the readers and the amount of reader variability. Regarding the variance estimates, the MRMC estimates are unbiased, whereas the naive estimates bracket the true variance and can be extremely biased.
© (2007) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Brandon D. Gallas and Gene A. Pennello "Pooling MRMC forced-choice data", Proc. SPIE 6515, Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment, 651506 (30 March 2007);


MRMC analysis of agreement studies
Proceedings of SPIE (March 24 2016)
Re use of pilot data and interim analysis of pivotal...
Proceedings of SPIE (March 10 2017)
Bootstrapped MRMC confidence intervals
Proceedings of SPIE (April 06 2005)
Comparing agreement measures
Proceedings of SPIE (March 24 2008)

Back to Top