|
We introduce an interesting interpretation of the ROC Curve that, subsequently, opens a new research paradigm. We define the "Diagnostician Operating Choice" (DOC) Curve to be the set of all (True Positive Probability/True Negative Probability) or ("skill in diseased population"/"skill in non-diseased population" when considered from the diagnostician's perspective) options made available to a particular radiologist when interpreting a particular diagnostic technology. The DOC Curve is, thus, the choice set presented to the diagnostician by their interaction with the technology. This new paradigm calls for tools that can measure the particular choice set of any particular individual radiologist interpreting a particular technology when applied in a particular clinical setting. Fundamental requirements for this paradigm are for the DOC Curve to be unique to individuals and constant across similar experimental conditions. To investigate constancy, we analyzed data from a reading study of 10 radiologists. Each radiologist interpreted the same set of 148 screening mammograms twice using a modified version of BI-RADS. ROC Curves for each radiologist were computed and compared between the two reading occasions with the CORROC2 program. None of the areas were statistically significantly different (p<0.05), providing confirmation (but not proof) of constancy across the two reading conditions. The DOC Curve paradigm suggests new areas of research focusing on the behavior in individuals interacting with technology. A clear need is for more efficient estimation of individual DOC Curves based on limited case sets. Paradoxically, the answer to this last problem might lie in using large population-based ("MRMC") studies to develop highly efficient and externally validated standardized testing tools for assessment of the individual.
|
