Translator Disclaimer
24 June 1998 Variations in measured performance of CAD schemes due to database composition and scoring protocol
Author Affiliations +
There is now a large effort towards developing computer- aided diagnosis (CAD) techniques. It is important to be able to compare performance of different approaches to be able to determine which ones are the most efficacious. There are currently a number of barriers preventing meaningful (statistical) comparisons, two of which are discussed in this paper: database composition and scoring protocol. We have examined how the choice of cases used to test a CAD scheme can affect its performance. We found that our computer scheme varied between a sensitivity of 100% to 77%, at a false-positive rate of 1.0 per image, with only 100% change in the composition of the database. To evaluate the performance of a CAD scheme the output of the computer must be graded. There are a number of different criteria that are being used by different investigators. We have found that for the same set of detection results, the measured sensitivity can be between 40 - 90% depending on the scoring methodology. Clearly consensus must be reached on these two issues in order for the field to make rapid progress. As it stands now, it is not possible to make meaningful comparisons of different techniques.
© (1998) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Robert M. Nishikawa and Laura M. Yarusso "Variations in measured performance of CAD schemes due to database composition and scoring protocol", Proc. SPIE 3338, Medical Imaging 1998: Image Processing, (24 June 1998);

Back to Top