|
In this paper, we propose a methodology for evaluating whether the use of CAD is effective for any given reader or
case, first analyzing the results of readers' judgments (0 or 1) by the technique known as analysis of bias-variance
characteristics (BVC)1,2, then by combining this with ROC analysis, elucidating the internal structure of the ROC curve.
The mean and variance are first calculated for the situation when multiple readers examine a medical image for a single
case without CAD and with CAD, and assign the values 0 and 1 to their judgment of whether abnormal findings are
absent or present or whether the case is normal or abnormal. The mean of these values represents the degree of bias
from the true diagnosis for the particular case, and the variance represents the spread of judgments between readers.
When the relationship between the two parameters is examined for several cases with differing degrees of diagnostic
difficulty, the mean (horizontal axis) and variance (vertical axis) show a bell-shaped relation. We have named this
typical phenomenon arising when images are read, the bias-variance characteristic (BVC) of diagnosis. The mean of the
0 and 1 judgments of multiple readers is regarded as a measure of the confidence level determined for the particular
case. ROC curves were drawn by usual methods for diagnoses made without CAD and with CAD. From the difference
between the TPF obtained without CAD and with CAD for the same FPF on the ROC curve, we were able to quantify
the number of cases, the total number of readers, and the total number of cases for which CAD support was beneficial.
To demonstrate its usefulness, we applied this method to data obtained in a reading experiment that aimed to evaluate
detection performance for abnormal findings and data obtained in a reading experiment that aimed to evaluate
diagnostic discrimination performance for normal and abnormal cases. We analyzed the internal structure of the ROC
curve produced when all cases were included, and showed that there is a relationship between the degree of diagnostic
difficulty of the case and the benefit of CAD support and demonstrated that there are patients and readers for whom
CAD is of benefit and those for whom it is not.
|
