Medical images contain information (in two, three, and four dimensions) that is important to the clinician in diagnosis, evaluation, and treatment. New methodologies and tools make it possible to extract, interpret, integrate, and display that information very effectively. This paper describes some of the application areas, gives examples of the tools now available and under development, identifies the benefits and costs of the new approaches, examines the issues in evaluation and validation, and looks at opportunities for further improvements.

Significant advances are being made in the area of automated medical image analysis. Part of the progress is due to the general advances being made in the types of algorithms used to process images and perform various detection and recognition tasks. A more important reason for this growth in medical image analysis processes, may be due however to a very different reason. The use of computer workstations, digital image acquisition technologies and the use of CRT monitors for display of medical images for primary diagnostic reading is becoming more prevalent in radiology departments around the world. With the advance in computer- based displays, however, has come the realization that displaying images on a CRT monitor is not the same as displaying film on a viewbox. There are perceptual, cognitive and ergonomic issues that must be considered if radiologists are to accept this change in technology and display. The bottom line is that radiologists' performance must be evaluated with these new technologies and image analysis techniques in order to verify that diagnostic performance is at least as good with these new technologies and image analysis procedures as with film-based displays. The goal of this paper is to address some of the perceptual, cognitive and ergonomic issues associated with reading radiographic images from digital displays.

We propose a collection of robust algorithms for the segmentation of cell images from Papanicolaou stained cervical smears (`Pap' smears). This problem is deceptively difficult and often results on laboratory datasets do not carry over to real world data. Our approach is in 3 parts. First, we segment the cytoplasm from the background using a novel method based on the Wilson and Spann multi-resolution framework. Second, we segment the nucleus from the cytoplasm using an active contour method, where the best contour is found by a global minimization method. Third, we implement a method to determine a confidence measure for the segmentation of each object. This uses a stability criterion over the regularization parameter (lambda) in the active contour. We present the results of thorough testing of the algorithms on large numbers of cell images. A database of 20,120 images is used for the segmentation tests and 18,718 images for the robustness tests.

Much of the effort in recent years has been directed to the development of algorithms and protocols that address various potential sources of artifact. A practical set of tools has emerged for both acquisition and reconstruction. Foremost in the available techniques has been the development of techniques for the simultaneous acquisition of transmission and emission data with subsequent reconstruction using accelerated maximum likelihood reconstruction. The available of measured transmission data has also aided the development of more rigorous models for scatter and possible techniques for correction of partial volume errors. Acceleration has provided a feasible means for incorporating more rigorous, albeit more complex, models of the emission and detection processes, including distance-dependent resolution and patient motion. In addition, combination of data from anatomical and functional modalities has not only aided in visual identification of clinical features, but also has permitted enhanced reconstruction based on the complementary information.

In tomography, activity recorded on cameras provides acquired counts from which the medical image of interest is reconstructed. These acquired data are only indirectly related to the source distribution within the body. This latter distribution can only be inferred indirectly (by mathematical analysis) from the camera projection data.

The rapidly growing use of digital images for medical applications results in an inevitable need to compress the data, to gain cost and time benefits in storage and transmission. This paper reviews some different approaches taken in investigating and applying medical image compression, including a discussion of fundamental quality and performance issues in the evaluation of image compression techniques. Three classes of properties related to image compression techniques are identified: mathematical, psychovisual and task-oriented. Subsequently, some aspects of the different approaches are drawn together in the concept of selectable quality compression, which is advocated as a promising direction for this application area.

Image registration techniques spatially register clinical images of patients performed either at different times with the same modality (intra-modality) or with different modality (inter-modality), to facilitate assessment of change and to take full advantage of the frequently complementary information provided by the imaging modalities. Inter-subject registration permits comparison to normal data bases and averaging of data from several subjects to improve statistical significance. Image registration is well established in the brain since the skull limits deformation of the brain between studies and the use of rigid body transformation can usually be justified for intra-subject registrations. A large range of image registration algorithms, ranging from completely manual to fully automatic, have been developed. These can be classified into external methods, which typically use fiducial markers, intrinsic techniques, which rely on the information contained in the patient image data and non- image based methods, which use information external to the data being registered. The technique of choice depends on the specific requirements of the application and it is unlikely that a single `best' technique can meet sometimes conflicting requirements (e.g. accuracy, speed, ease of use etc). While considerable progress has been made in image registration outside the brain, considerable challenges still remain. In this paper, we present the basic principles of image registration and practical issues arising from our experience with routine clinical use of image registration over several years.

The confocal scanning laser microscope (CSLM) is an exciting new tool in microscopy. It offers improved rejection of out- of-focus `noise' and greater resolution than conventional imaging. By integrating a computer into the system and generating digital image data files, a rapid way of storing, processing, and analyzing images is available to the user. The production of 3D reconstruction representations is easy and effective. The technique of optical sectioning and confocal optics has revolutionized epifluorescence microscopy, the CSLM providing a highly desirable link between conventional light microscopy and electron microscopy. The use of the CSLM in biomedical health sciences is considered in this paper and the functional basics of the instrument are discussed with reference to several important applications in research and diagnostic work, with illustrations from the numerous and continually increasing publications in the area. It is veritably a `solution in search of problems' as this short review demonstrates.

The Skin Polarprobe, an automated melanoma diagnosis system, has the potential to greatly improve the chances of early detection of skin cancers and help save the lives of melanoma victims. This paper will describe the development of this device from the initial proof-of-concept phase (Phase I) using digitized slide images, through the first prototype video-based system (Phase II), to the current status of the production prototype system. Because of commercial confidentiality, precise details of each step cannot be given. However, some of the technical difficulties at each step will be described and some general points about how to overcome them will be discussed. Many of these comments will be generic to any imaging application.

Knowledge-based image segmentation is defined in terms of the separation of image analysis procedures and representation of knowledge. Such architecture is particularly suitable for medical image segmentation, because of the large amount of structured domain knowledge. A general methodology for the application of knowledge-based methods to medical image segmentation is described. This includes frames for knowledge representation, fuzzy logic for anatomical variations, and a strategy for determining the order of segmentation from the modal specification. This method has been applied to three separate problems, 3D thoracic CT, chest X-rays and CT angiography. The application of the same methodology to such a range of applications suggests a major role in medical imaging for segmentation methods incorporating representation of anatomical knowledge.

Current use of medical imaging is far from satisfactory in terms of cost/effectiveness. To improve this, it is necessary to use database techniques. Different from a digital archive, a medical image database is capable of information abstraction, inferencing and reasoning. We will review current research and techniques world-wide, especially, the KMeD system developed in UCLA. We then address various issues in the development of medical image databases and present our work in the field. Many issues are also general concerns in medical imaging such as filtering, segmentation, registration, clustering, compression, reconstruction and visualization.

The development of data warehouses for the storage and analysis of very large corpora of medical image data represents a significant trend in health care and research. Amongst other benefits, the trend toward warehousing enables the use of techniques for automatically discovering knowledge from large and distributed databases. In this paper, we present an application design for knowledge discovery from databases (KDD) techniques that enhance the performance of the problem solving strategy known as case- based reasoning (CBR) for the diagnosis of radiological images. The problem of diagnosing the abnormality of the cervical spine is used to illustrate the method. The design of a case-based medical image diagnostic support system has three essential characteristics. The first is a case representation that comprises textual descriptions of the image, visual features that are known to be useful for indexing images, and additional visual features to be discovered by data mining many existing images. The second characteristic of the approach presented here involves the development of a case base that comprises an optimal number and distribution of cases. The third characteristic involves the automatic discovery, using KDD techniques, of adaptation knowledge to enhance the performance of the case based reasoner. Together, the three characteristics of our approach can overcome real time efficiency obstacles that otherwise mitigate against the use of CBR to the domain of medical image analysis.

In the past ten years, there has been a push to improve early detection of breast cancer by providing radiologists with computer assistance in assessing screening mammograms. A large variety of modern image analysis techniques have been proposed for automatically detecting and classifying anomalies in mammograms. Although much of the work has not been focused on the critical issues and there have been problems in comparing the performance of the various proposed techniques, substantial progress has been made. The field is now at the critical point of emerging from a state where the goal was to prove feasibility to a stage where the full potential of computer assistance can be realized. The three ingredients driving this transition are (1) recent studies which firmly establish a positive effect of computer assistance on assessing mammograms, (2) winning US FDA approval of the first commercial product for providing such assistance, and (3) the advent of direct digital image acquisition for screening mammography.

The latest trend in computer assisted mammogram analysis is reviewed and two new methods developed by the authors for automatic detection of microcalcifications (MCs) are presented. The first method is based on wavelet neurone feature detectors and ART classifiers while the second method utilized fuzzy rules for detection and grading of MCs.

The imaging of patients with cancer is of increasing importance. There is an increasing prevalence of tumors within a aging population and the success of cancer therapy has resulted in increased numbers of patients surviving cancer and a need to monitor the therapeutic response. Angiogenesis describes a fundamental process in the development of tumors whereby the growing malignancy appropriates its own blood supply from adjacent tissues. This process is essential for tumor growth and metastatic spread. Using a multi-disciplinary approach that builds from a basis of physiological and pathological principles to radiology and automated image analysis, the processes of angiogenesis can be visualized in vivo providing diagnostic and prognostic information for patients with cancer. This approach can also provide a more general template for the analysis of medical images.