We investigated effects of prevalence and case distribution on radiologist diagnostic performance as measured by area under the receiver operating characteristic curve (AUC) and sensitivity-specificity in lab-based reader studies evaluating imaging devices. Our retrospective reader studies compared full-field digital mammography (FFDM) to screen-film mammography (SFM) for women with dense breasts. Mammograms were acquired from the prospective Digital Mammographic Imaging Screening Trial. We performed five reader studies that differed in terms of cancer prevalence and the distribution of noncancers. Twenty radiologists participated in each reader study. Using split-plot study designs, we collected recall decisions and multilevel scores from the radiologists for calculating sensitivity, specificity, and AUC. Differences in reader-averaged AUCs slightly favored SFM over FFDM (biggest AUC difference: 0.047, SE = 0.023, p = 0.047), where standard error accounts for reader and case variability. The differences were not significant at a level of 0.01 (0.05/5 reader studies). The differences in sensitivities and specificities were also indeterminate. Prevalence had little effect on AUC (largest difference: 0.02), whereas sensitivity increased and specificity decreased as prevalence increased. We found that AUC is robust to changes in prevalence, while radiologists were more aggressive with recall decisions as prevalence increased.
The FDA recently completed a study on design methodologies surrounding the Validation of Imaging Premarket Evaluation and Regulation called VIPER. VIPER consisted of five large reader sub-studies to compare the impact of different study populations on reader behavior as seen by sensitivity, specificity, and AUC, the area under the ROC curve (receiver operating characteristic curve). The study investigated different prevalence levels and two kinds of sampling of non-cancer patients: a screening population and a challenge population. The VIPER study compared full-field digital mammography (FFDM) to screenfilm mammography (SFM) for women with heterogeneously dense or extremely dense breasts. All cases and corresponding images were sampled from Digital Mammographic Imaging Screening Trial (DMIST) archives. There were 20 readers (American Board Certified radiologists) for each sub-study, and instead of every reader reading every case (fully-crossed study), readers and cases were split into groups to reduce reader workload and the total number of observations (split-plot study). For data collection, readers first decided whether or not they would recall a patient. Following that decision, they provided an ROC score for how close or far that patient was from the recall decision threshold. Performance results for FFDM show that as prevalence increases to 50%, there is a moderate increase in sensitivity and decrease in specificity, whereas AUC is mainly flat. Regarding precision, the statistical efficiency (ratio of variances) of sensitivity and specificity relative to AUC are 0.66 at best and decrease with prevalence. Analyses comparing modalities and the study populations (screening vs. challenge) are still ongoing.
Purpose: The goal of this study was to begin quantifying the performance of a second generation diffraction
enhanced imaging (DEI) system designed to reduce imaging time from our first generation system.
Background: DEI, a phase contrast x-ray imaging modality, generates images with enhanced soft tissue contrast at a
lower dose than conventional radiography. Our group has previously reported on an x-ray tube-based DEI system,
but a substantial gap remained between the imaging time of that system and that of a clinical DEI system.
Method: A high power, rotating anode x-ray tube was integrated into this second generation DEI system along with
an image intensifier to improve photon counting efficiency. Images of phantoms were acquired using a tube power
of 10 kW (125 kVp, 80 mA) at the silicon  reflection with the analyzer crystal at its half-reflectivity point.
Results: Our preliminary results show comparable image contrast to the first generation DEI system. Imaging time
was reduced by a factor four and x-ray-on time was reduced by a factor of sixty from the initial prototype system.
Conclusions: These early results from our second generation diffraction enhanced imaging system show significant
reduction in imaging time with preservation of DEI contrast.
Conventional mammographic image contrast is derived from x-ray absorption, resulting in breast structure visualization
due to density gradients that attenuate radiation without distinction between transmitted and scattered or refracted x-rays.
This leads to image blurring and contrast reduction, hindering the early detection of small or otherwise occult cancers.
Diffraction enhanced imaging (DEI) allows for dramatically increased contrast with decreased radiation dose compared
to conventional mammographic imaging due to monochromatic x-rays, its unique refraction-based contrast mechanism
and excellent scatter rejection. However, a lingering drawback to the clinical translation of DEI has been the requirement
for synchrotron radiation. Our laboratory developed a DEI prototype (DEI-PR) utilizing a readily available Tungsten xray
tube source and traditional DEI crystal optics, providing soft tissue images at 60keV. To demonstrate the clinical
utility of our DEI-PR, we acquired images of full-thickness human breast tissue specimens on synchrotron-based DEI,
DEI-PR and digital mammography systems. A reader study was designed to allow unbiased assessment of system
performance when analyzing three systems with dissimilar imaging parameters and requiring analysis of images
unfamiliar to radiologists. A panel of expert radiologists evaluated lesion feature visibility and histopathology correlation
after receiving training on the interpretation of refraction contrast mammographic images. Preliminary data analysis
suggests that our DEI system performed roughly equivalently with the traditional DEI system, demonstrating a
significant step toward clinical translation of this modality for breast cancer applications.
One of the advantages of digital mammography is the ability to acquire a mammography image with a larger contrast range. With this advantage comes the tradeoff of how to display this larger contrast range. Laser printed film, and video display both have smaller dynamic ranges than standard mammography film-screen systems. This work examines performance and preference studies for display processing methods for digital mammograms.
We are using a beam port at the National Synchrotron Light Source Facility at Brookhaven National Laboratory as a source of monoenergetic photons. The photon source is radiation from a bending magnet on the x-ray storage ring and provides a usable x-ray spectrum from 5 keV to over 50 keV. A tunable crystal monochromotor is used for energy selection. The beam is 79 mm wide and 0.5 mm high. We imaged the ACR mammography phantom and a contrast-detail phantom using a phosphor plate as the imaging detector. Phantom images were obtained at 16, 18, 20, and 22 keV. Phantom thickness varied from 15 mm to 82 mm. These images were compared to images obtained with a conventional dedicated mammography unit. Subjective preliminary results show that image contrast of the monoenergetic images is similar to those obtained from the conventional x-ray source with somewhat sharper and cleaner images from the monoenergetic source. Quantitative analysis shows that the monoenergetic images have improved contrast compared to the polyenergetic derived images. Entrance skin dose measurements show a factor of 5 to 10 times less radiation for the monoenergetic images with equivalent or better contrast. Although there remain a number of technical problems to be addressed and much more work to be done, we are encouraged to further explore the use of monoenergetic imaging.
Real-time volume rendering of medical image datasets on commercial hardware became possible in 1993. We have developed an application, SeeThru, that allows real-time volume visualization under the interactive control of the physician. This ability enables the physician to look inside of the patient's body to visually comprehend the information from radiological procedures, resulting in improved treatment planning. We report on preliminary results from two areas: (1) cardiothoracic surgical planning from spiral computed tomography (CT) and (2) staging of breast cancer from magnetic resonance imaging (MRI). We compared different rendering methods (projection, maximum intensity projection, opacity blended, and opacity combined with gradient blended) and chose opacity blending as the most effective for both applications. In cardiothoracic surgical planning experiment we found the ability to interactively control and view 3D direct volume visualizations resulted in improvements in surgical plans and in the surgeon's confidence in the plan. In the MR breast experiment we found that 3D visualization of the subtraction images improved comprehension and identification of tumor lesions difficult to appreciate on mammograms. Overall, we believe that interactive, real-time volume rendering significantly adds to clinical understanding and improves treatment planning for the patient.
Proc. SPIE. 1808, Visualization in Biomedical Computing '92
KEYWORDS: Image visualization, Biomedical optics, Visual process modeling, Visualization, Quality measurement, Medical imaging, Image quality, Image enhancement, Human vision and color perception, Performance modeling
We have developed and are applying two methods of image quality assessment with the aim of optimizing contrast enhancement parameter settings and evaluating competing methods. Our first approach uses observer studies employing psychophysical methods and a realistic clinical task; the second incorporates a model of human vision in a computer simulation of the performance of an observer.
We are investigating how radiologists's readings of standard intensity windowed (IW) chest computed tomography (CT) films compare with readings of the same images processed with contrast limited adaptive histogram equalization (CLAHE). Previously reported studies where CLAHE has been tested have involved detection of computer generated targets in medical images. Our study is designed to evaluate CLAHE when applied to clinical material and to compare the diagnostic information perceived by the radiologists from CLAHE processed images to that from the conventional IW images. Our initial experiment with two radiologists did not yield conclusive results, due in part, to inadequate observer training prior to the experiment. The initial experimental protocol was redesigned to include more in-depth training. Three new radiologist observers were recruited for the follow-up study. Results from the initial study are reviewed and the follow-up study is presented. In the new study we find that while CLAHE and IW are not statistically significantly different overall, there are specific clinical findings where the radiologists were less comfortable reading CLAHE presentations. Advantages and disadvantages of using CLAHE as a replacement or as an adjunct to IW are discussed.
For the last four years, the UNC FilmPlane project has focused on constructing a radiology workstation facilitating CT interpretations equivalent to those with film and viewbox. Interpretation of multiple CT studies was originally chosen because handling such large numbers of images was considered to be one of the most difficult tasks that could be performed with a workstation. The authors extend the FilmPlane design to address mammography. The high resolution and contrast demands coupled with the number of images often cross- compared make mammography a difficult challenge for the workstation designer. This paper presents the results of preliminary work with workstation interpretation of mammography. Background material is presented to justify why the authors believe electronic mammographic workstations could improve health care delivery. The results of several observation sessions and a preliminary eyetracker study of multiple-study mammography interpretations are described. Finally, tentative conclusions of what a mammographic workstation might look like and how it would meet clinical demand to be effective are presented.