Translator Disclaimer
1 January 2007 Snapshot hyperspectral imaging in ophthalmology
Author Affiliations +
Abstract
Retinal imaging spectroscopy can provide functional maps using chromophore spectra. For example, oxygen saturation maps show ischemic areas from diabetes and venous occlusions. Obtaining retinal spatial-spectral data has been difficult due to saccades and long data acquisition times (>5 s). We present a snapshot imaging spectrometer with far-reaching applicability that acquires a complete spatial-spectral image cube in ~3 ms from 450 to 700 nm with 50 bands, eliminating motion artifacts and pixel misregistration. Current retinal spectral imaging approaches are incapable of true snapshot operation over a wide spectral range with a large number of spectral bands. Coupled to a fundus camera, the instrument returns true color retinal images for comparison to standard fundus images and for image validation while the patient is still dilated. Oxygen saturation maps were obtained with a three-wavelength algorithm: for healthy subjects arteries were ~95% and veins 30 to 35% less. The instrument is now undergoing clinical trials.

Hyperspectral imaging devices are common in remote sensing reconnaissance as well as biomedical applications.1 Typically, these instruments are point or slit imagers and must scan the scene temporally to build up a two-dimensional image, or use bandpass filters to scan the scene spectrally. More specific types, including Hadamard encoding slit spectrometers,2 liquid crystal and acousto-optic tunable filters,3 Fourier transform spectrometers,4 spectral-temporal scanning,5 and more recently volume holographic methods6; all scan to build up the spatial-spectral cube of information. In this article the computed tomographic imaging spectrometer (CTIS) is used. It captures both spatial and spectral information in a single frame. Its acquisition time is only constrained by the exposure time of the camera (typically on the order of milliseconds) and required signal-to-noise ratio. There are no moving parts or narrow-band filters, and nearly all collected light (about 70%) is passed to the detector at all times.

Although CTIS has been used in the past for stellar classification,7 fluorescence microscopy,8 and geology measurements,9 this report is on the groundbreaking use of CTIS for functional mapping of the human retina. This application is ideal for exploring the true potential for CTIS technology since the eye is constantly moving, requiring snapshot camera operation. Hyperspectral imaging offers functional rather than structural information, such as optical coherence tomography, and provides complementary information for the clinician. Our results show how hemoglobin spectral signatures provide both qualitative and quantitative oxygen saturation maps. This data offers the possibility of monitoring retinal ischemia from either systemic diseases such as diabetes or from localized retinal arterial and vascular occlusions—the leading causes of untreatable blindness.

Retinal hyperspectral imaging offers the ability to capture in vivo metabolic and physiologic information using chromophore spectra to classify tissues and quantify cellular metabolites.10, 11, 12 All of the software tools developed for remote sensing as well as signal and image processing can be applied to the hyperspectral data obtained. Clinical trials are underway at the Doheny Eye Institute, Keck School of Medicine at the University of Southern California to establish a significant patient database.

Current ophthalmic hyperspectral imaging systems13, 14 typically require long exposure times 5s , and therefore are plagued with motion artifacts from rapid eye movements (saccades). Both major and minor saccades occur frequently enough to significantly affect both band-sequential and line-scanning instruments. These data then require significant and laborious postprocessing and registration. Image registration compromises either spatial or spectral resolution, depending on the type of system used. One does not know which wavelengths are affected, cannot be registered, or simply miss the relevant spatial features used for registration in a clinical setting until after the patient has left. For retinal imaging especially, postacquisition registration has artifacts from both changes in the lens-eye geometry and tissue movement from arterial pulses between successive images. Furthermore, patients cannot fixate long enough or accept the high illumination levels required to keep exposure times short enough. The true snapshot capability of the CTIS imager avoids all these problems and provides an easy, patient-friendly way to obtain retinal spectral data. Guidelines of the Association for Research in Vision and Ophthalmology for human investigation were followed, and our Human Investigation Committee approved the study protocol.

CTIS captures the spatial and spectral information of the retina by imaging the scene through a two-dimensional grating disperser as illustrated in Fig. 1 . This produces multiple, spectrally dispersed images of the retina that are recorded by a focal plane array (FPA). From the captured intensity pattern, computed-tomography algorithms are used to reconstruct the scene into a “cube” of spatial ( x and y ) and spectral (wavelength) information. The image cube in wavelength space is then reconstructed from a single image. Note that the basic CTIS design uses just two lenses and a focal plane detector. The CTIS instrument concept originated in Japan15, 16 and Russia17 and has been significantly advanced to maturity by a group at the Jet Propulsion Laboratory (JPL) in Pasadena18, 19, 20 and one at the University of Arizona. 21, 22, 23, 24 The physics and the mathematics describing CTIS imaging and data cube reconstruction are similar to that of positron-emission tomography, hence the name “computed tomography” imaging spectrometer.

Fig. 1

Computed tomographic imaging spectrometer (CTIS) schematic. An image from any primary optical system is the input for the CTIS. The diffractive grating is in collimated space and disperses the image in two dimensions. A second lens re-images the pattern onto the image detector.

014036_1_001701jbo1.jpg

The specific type of diffractive optical element used is called a computer-generated hologram (CGH). It operates at multiple orders, creating an array of images on the sensor. Development of techniques for fabricating the grating with electron-beam lithography has been the main driver in the development of this instrument.25 The CGHs are composed of cells of square pixels (typically 8×8 pixels/cell) that are arrayed to form a 2-D grating. The pixel depths are designed using a CTIS-specific iterative optimization algorithm such that the dispersion order intensities are nearly equal after distribution across the focal plane array. This maximizes the overall system signal-to-noise ratio by avoiding saturation in a single diffraction order. An analog direct-write electron-beam lithography technique was developed to accurately fabricate a CGH. This technique requires only a single exposure followed by iterative development. It allows the production of fine-featured analog surface-relief profiles in thin films of E-beam resist with a depth error less than 5%.

Figure 2 shows how the various orders are dispersed across the focal plane as well as spectra derived for an image scene. It is important to note that each image is not simply composed of single wavelengths; spatial and spectral information from each object pixel is multiplexed over the entire array. Figure 2 shows how the spectrum of a single pixel is distributed by the diffractive disperser. Hence, a single image contains all the information required to reconstruct the spectral image cube. Note that there is a zero-order image for focusing—a difficult task for many spectral imagers.

Fig. 2

Computed tomographic imaging spectrometer (CTIS). The observed scene in (a) consists of three-color LEDs, a HeNe laser spot, and an eight-segment indicator. (b) The resulting image from the monochrome focal plane. A zero-order image, (c) can be used for focusing as well as providing an initial starting point for the reconstruction of the image cube. Recovered spectra are shown in (d). The image cube has been reconstructed with 32 10-nm wide bands with an image that is 64×64 pixels in size.

014036_1_001701jbo2.jpg

A calibration matrix was necessary to perform the retinal reconstruction; it was obtained by measuring the location on the image plane of pixels from the object plane at different wavelength with a movable fiber optic coupled to a monochromator. The efficiencies, spot centroids, and point spread functions of all the diffraction orders are measured at all wavelengths in the operating band. This data is used to build a “system transfer matrix” of field-stop-to-detector connection coefficients. Once the system transfer matrix was known, the images of the retina were reconstructed to obtain their spectra. Following Descour 21 the iterative expectation-maximization (EM) reconstruction algorithm was used. This algorithm was developed for reconstructing positron-emission tomography medical images,26, 27 but other emission tomography algorithms are applicable as well. This estimation approximates the spatial and spectral information using voxel basis functions, which define the solution space for accurate reconstructions. Although there are various forms of iterative algorithms, EM is used due to its robust solutions and proven reliability. The reconstruction starts with an initial condition for the scene which results in a predicted detector image. Corrections are then made to the scene based on back projection of the ratio of the measured detector image and predicted detector images. This process is repeated iteratively until the predicted detector image matches the measured detector image to a desired error tolerance. Typically 10 to 20 iterations were required for good retinal reconstructions. National Institute of Standards and Technology (NIST) traceable reflectance color targets were used to verify the accuracy of the CTIS reconstruction process. Comparison of the CTIS spectral reflectance shows agreement to the NIST standards within a few percent over 450 to 750nm .

The latest version of CTIS reported here replaces the usual monochrome camera with a digital color CCD (Qimaging © Retiga 4000R) as the detector. Imposing spatial or spectral modulation on the image scene significantly improves the spectral reconstructions.20 The Bayer color filters on the camera pixels that provide each RGB plane act to provide spectral modulation. It was found that using the color image as the initial condition for the reconstructions significantly reduces reconstruction time by decreasing the number of iterations required. This is due to the fact that the color image already contains some spectral information as the initial “guess.” Figure 3 shows spectral calibration data for the CTIS taken with the color camera system. Results are shown in Fig. 3a for NIST traceable color targets in which the CTIS results match the standard to a few percent across the spectral range. A second set of standards (rare earth), shown in Fig. 3b, allows verification with high-frequency spectral components.

Fig. 3

CTIS spectral calibration. (a) Results for color targets. (b) Results for rare earth calibration targets. In each case, the NIST standard is the solid line while the symbols are reflectance measured with a CTIS.

014036_1_001701jbo3.jpg

A Zeiss ff450+ fundus camera was used as the objective assembly to image the retina of the human test subjects. An example of a raw image from the imager is shown in Fig. 4 . A filter matched to the camera flash bulb spectra optimized the dynamic range of the silicon detector from 500 to 600nm . The fundus objective had a large 24 ×36mm image plane. Demagnification (17.2×) was needed in order to compensate for this large size and was accomplished using a 2× tele-extender along with a 180-mm commercial-off-the-shelf (COTS) lens assembly for collimation. The field stop was set at 11×11mm and combined with a 50-mm COTS re-imaging lens reduced the field of view to 18deg with a Fundus camera setting of 50deg , with minimal vignetting. This field of view converts to an image plane at the detector across 208×208 pixels. The Fundus camera has three different settings for the field of view and each maps a higher resolution image across the pixel space of the zeroth order CTIS. The Fundus camera settings of 50, 30, and 20deg correspond to 18, 10.8, and 7.2deg , respectively, with our optics and field stop. Small vessels ( <75μm in diameter) are best visualized at the 7.2-deg setting while larger vessels ( > 150μm in diameter) can be easily imaged at the 18-deg setting. Although it has a smaller field of view, the 18-deg setting still images both the optic disk and macula together, allowing the clinician to easily compare with typical retinal images. Due to the large numerical aperture of the fundus objective, a field lens is implemented to counterbalance the spatial variance due to the vignetted high-angle rays. The field lens was placed just outside the field stop so that any dust on the lens would be out of focus, making a negligible change to the demagnification.

Fig. 4

Color CTIS diffraction pattern. An image of an optic disk and the surrounding vascular structure is shown. The field of view is 12deg (30-deg Fundus camera setting). The maximum field of view obtained with this system is 18deg , which is enough to show the macula and optic disk in one shot. Note the radial dispersion of the images.

014036_1_001701jbo4.jpg

This article reports on the co-registered image and associated oximetry unmixed reflectance spectra for different parts of the human retina. Incident illumination is spectrally modified due to scattering and absorption contributions within the eye as approximately described by Lambert-Beer’s law. Previously measured extinction coefficients are used for oxy-hemoglobin and deoxy-hemoglobin28 to calculate absorption and scattering coefficients due to the turbid media of the layered retinal structure. For this calculation both scattering and absorption due to the aqueous and vitreous humor are neglected. Our model assumes all scattering and absorption are only due to the vascular surface and underlying RPE layers of the retina. To a first approximation, the vascular structure’s absorption is given by the linear model, which simply multiplies the absorption and scattering, respectively, per unit length and per unit concentration with the absorption and scattering concentration through some retinal tissue depth. The goal is to find the percentage of oxygen saturation based on a weighted linear model. This model is constructed based on previous literature using typical molar concentration (150 g/l) and molecular weight (64,500 g/mole) for healthy individuals. In the generalized linear oximetry model the natural logarithm is used with the ratio of the signal versus background response for capillary vessels. This is purely based on Lambert-Beer’s law where each response is added in the exponent of the exponential function.

Oxygen saturation algorithms were used based on previous multiwavelength oximetry models from slit scanning spectrometers. Specifically, a three-wavelength oximetry model was used to localize the saturation values spatially for each scene. A totally unsupervised approach was taken in order to show how saturation values, localized at positions known to be a vessel, can be found for each dataset. The model is described elsewhere.29, 30, 31 The output of this function allows a spatial-saturation map. The local oxygen saturation is related to the health of the particular area of an individual’s retina.

Reflectance from wavelengths shorter than 500nm is adversely affected by light scattering in ocular media and erythrocytes.32 For wavelengths longer than 585nm , a drastic decrease in hemoglobin extinction coefficients causes light scattering and absorption to dominate in deeper retinal tissues and intervening ocular media. In order to properly perform three-wavelength oximetry, the following optimization procedure was employed. Three wavelengths were randomly chosen from the imaged wavelength interval of 454 to 646nm . Subsequently these three wavelengths were modified according to a multivariate optimization algorithm with a spectral resolution of 1nm (the imaged hyperspectral data cube is in increments of 4nm between hyperspectral planes). Linear interpolation of the hyperspectral planes from the three wavelengths provided hyperspectral data at in-between wavelengths for each iteration step of the optimization process. Each hyperspectral plane used for the interpolation was normalized and white-corrected (divided by the measured spectra of the flashbulb). The respective normalization factors were subject to the same multivariate optimization process used for the wavelengths, resulting in a 3+6 parameter optimization. The fitness function for this optimization process was the necessary condition that in order to properly perform three-wavelength oximetry calculations, the oximetry value for each mapped pixel in the image had to be within the physically meaningful interval [0,1]. Upon convergence of the optimization procedure, the optimized set of the three wavelengths together with the six normalization factors for the hyperspectral planes necessary for the interpolation were determined and the oximetry values for the entire image were calculated.

To suppress background scatter a Monte Carlo optimization routine was used in combination with the three-wavelength model already described. Starting again with the oxy-hemoglobin and deoxy-hemoglobin extinction coefficients and three wavelengths, a stochastic search of the local molar concentration space is given by

Eq. 1

φλ=iGpiλyiλpiλφ2,pi2[0,1]
where p is the probability of an event occurring at the correct value and y is the perturbing search vector spanning the region of molar concentration of interest. The output yields the oxygen saturation map given by φ , since Eq. 1 is performed upon the entire cube simultaneously. Furthermore, the angular dependency of the bulb illumination is taken out using a Gaussian illumination approximation. The Gaussian function is adjusted based on the tendency of the bulb illumination to vary spatially for each image around the edges. This is due to operator pointing motion and the complex dynamic function of the human lens-eye system.

Two volunteers were both considered healthy but with an age difference of 30 years. Figure 5 shows a 12-deg image of the optic disk for volunteers 1 and 2, respectively. The results show a clear distinction between veins, arteries, and background. Some mixture of saturation value distribution is expected as has been found in past line-scanning instruments.32, 33 Regions within vessel capillaries agree well with the 30 to 35% oxygen saturation difference expected for healthy veins and arteries. 32, 33, 34, 35 The saturation for most of the background spatial locations in between the capillary regions shows a tendency to be within the 90 to 100% regime. This is consistent with the subjects being healthy.

Fig. 5

Spatial oxygen saturation maps. (a) Healthy male (29 years old) oxygen saturation map. Vascular separation from the background is seen as well as reasonable saturation values for veins versus arteries. (b) Zero-order color image. (c) Healthy male (58 years old) oxygen saturation map. (d) Zero-order color images.

014036_1_001701jbo5.jpg

The method for acquisition of retinal spectral images demonstrated in this paper has wide applications in any biomedical imaging where patient motion, arterial pulses, or patient comfort must be considered, such as bedside care. One implementation of a snapshot CTIS is inexpensive, is hand held, and does not require a computer to take data. This opens the door for the routine use of spectral imaging in areas outside of biomedicine, for example, forensics, art conservation, color, and field use in archaeology and agriculture.

Acknowledgments

The work described in this publication was carried out at the Jet Propulsion Laboratory, California Institute of Technology under a contract with the National Aeronautics and Space Administration and in addition supported by Reichert, Inc.

References

1. 

T. Vo-Dinh, Biomedical Photonics Handbook, CRC Press, Boca Raton, FL (2002). Google Scholar

2. 

R. Riesenberg, and U. Dillner, “HADAMARD imaging spectrometer with micro slit matrix,” Proc. SPIE, 3753 203 –213 (1999). 0277-786X Google Scholar

3. 

H. R. Morris, C. C. Hoyt, and P. J. Treado, “Imaging spectrometers for fluorescence and Raman microscopy: acousto-optic and liquid-crystal-tunable filters,” Appl. Spectrosc., 48 857 –866 (1994). https://doi.org/10.1366/0003702944029820 0003-7028 Google Scholar

4. 

T. H. Chao, H. Zhou, X. Xia, and S. Serati, “Hyperspectral imaging using electro-optic Fourier transform spectrometer,” Proc. SPIE, 5437 163 –170 (2004). 0277-786X Google Scholar

5. 

J. E. Murguia, T. D. Reeves, J. M. Mooney, W. S. Ewing, and F. D. Shepherd, “A compact visible/near infrared hyperspectral imager,” Proc. SPIE, 4028 457 –468 (2000). 0277-786X Google Scholar

6. 

W. Liu, G. Barbastathis, and D. Psaltis, “Volume holographic hyperspectral imaging,” Appl. Opt., 43 3581 –3599 (2004). https://doi.org/10.1364/AO.43.003581 0003-6935 Google Scholar

7. 

J. F. Scholl, E. K. Hege, M. L. Hart, D. O’Connell, W. R. Johnson, and E. L. Dereniak, “Evaluations of classification and spectral unmixing algorithms using ground based satellite imaging,” Proc. SPIE, 6233 807 –819 (2006). 0277-786X Google Scholar

8. 

F. Ford, D. Descour, and R. Lynch, “Large-image-format computed tomography imaging spectrometer for fluorescence microscopy,” Opt. Express, 9 (9), 444 –453 (Oct. 2001). 1094-4087 Google Scholar

9. 

W. R. Johnson, D. W. Wilson, and G. Bearman, “MEMS integration with an all-reflective SWIR snapshot hyperspectral imager,” Proc. SPIE, 6302 (2006). 0277-786X Google Scholar

10. 

J. B. Hickam, R. Frayser, and J. C. Ross, “A study of retinal venous blood oxygen saturation in human subjects by photographic means,” Circulation, 27 375 –385 (1963). 0009-7322 Google Scholar

11. 

J. B. Hickam and R. Frayser, “Studies of the retinal circulation in man: observations on vessel diameter, arteriovenous oxygen and mean circulation time,” Circulation, 33 302 –316 (1966). 0009-7322 Google Scholar

12. 

J. S. Tiedeman, S. E. Kirk, and S. Srinivas, “Retinal oxygen consumption during hyperglycemia in patients with diabetes without retinopathy,” Ophthalmology, 105 (1), 31 –36 (1998). https://doi.org/10.1016/S0161-6420(98)71029-1 0161-6420 Google Scholar

13. 

J. M. Beach, K. J. Schwenzer, and S. Srinivas, “Oximetry of retinal vessels by dual-wavelength imaging: calibration and influence of pigmentation,” J. Appl. Physiol., 86 (2), 748 –758 (1999). 8750-7587 Google Scholar

15. 

T. Okamoto and I. Yamaguchi, “Simultaneous acquisition of spectral image information,” Opt. Lett., 16 1277 –1279 (1991). 0146-9592 Google Scholar

16. 

T. Okamoto, A. Takahashi, and I. Yamaguchi, “Simultaneous acquisition of spectral and spatial intensity distribution,” Appl. Spectrosc., 47 1198 –1202 (1993). https://doi.org/10.1366/0003702934067810 0003-7028 Google Scholar

17. 

F. V. Bulygin, G. N. Vishnyakov, G. G. Levin, and D. V. Karpukhin, “Spectrotomography—a new method of obtaining spectrograms of 2-D objects,” Opt. Spectrosc., 71 561 –563 (1991). 0030-400X Google Scholar

18. 

W. R. Johnson, D. W. Wilson, G. H. Bearman, and J. Backlund, “An all-reflective computed tomography imaging spectrometer,” Proc. SPIE, 5660 88 –97 (2004). 0277-786X Google Scholar

19. 

W. R. Johnson, D. W. Wilson, and G. Bearman, “All-reflective computed tomographic imaging spectrometer for applications in ultraviolet and infrared,” Opt. Lett., 30 1 –3 (2005). https://doi.org/10.1364/OL.30.000510 0146-9592 Google Scholar

20. 

W. R. Johnson, D. W. Wilson, and G. Bearman, “Spatial-spectral modulating snapshot hyperspectral imager,” Appl. Opt., 45 1898 –1908 (2006). https://doi.org/10.1364/AO.45.001898 0003-6935 Google Scholar

21. 

M. R. Descour and E. L. Dereniak, “Computed-tomography imaging spectrometer: experimental calibration and reconstruction results,” Appl. Opt., 34 4817 –4826 (1995). 0003-6935 Google Scholar

22. 

M. R. Descour, C. E. Volin, E. L. Dereniak, T. M. Gleeson, M. F. Hopkins, D. W. Wilson, and P. D. Maker, “Demonstration of a computed-tomography imaging spectrometer using a computer-generated hologram disperser,” Appl. Opt., 36 3694 –3698 (1997). 0003-6935 Google Scholar

23. 

C. Volin, “MWIR spectrometer operating theory,” (2000) Google Scholar

24. 

K. Hege, D. O’Connell, W. Johnson, S. Basty, and E. L. Dereniak, “Hyperspectral imaging for astronomy and space surveillance,” Proc. SPIE, 5159 380 –391 (2003). 0277-786X Google Scholar

25. 

D. W. Wilson, R. E. Muller, P. M. Echternach, and J. P. Backlund, “Electron-beam lithography for micro- and nano-optical applications,” Proc. SPIE, 5720 68 –77 (2005). 0277-786X Google Scholar

26. 

L. A. Sheep and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Trans. Med. Imaging, MI-1 113 –122 (1982). 0278-0062 Google Scholar

27. 

J. P. Garcia and E. L. Dereniak, “Mixed-expectation image-reconstruction technique,” Appl. Opt., 38 3745 –3748 (1999). 0003-6935 Google Scholar

28. 

, by Scott Prahl, Oregon Medical Laser Center, 15 Dec 1999. http://omlc.ogi.edu/spectra/hemoglobin/ Google Scholar

29. 

R. N. Pittman and B. R. Duling, “A new method for the measurement of percent oxyhemoglobin,” J. Appl. Physiol., 38 2 (1975). 0021-8987 Google Scholar

30. 

F. C. Delori, “Non-invasive technique for oximetry of blood in retinal vessels,” Appl. Opt., 27 1113 –1125 (1988). 0003-6935 Google Scholar

31. 

M. H. Smith, “Optimum wavelength combinations for retinal vessel oximetry,” Appl. Opt., 38 258 –267 (1999). 0003-6935 Google Scholar

32. 

D. Schweitzer, M. Hammer, and J. Kraft, “In vivo measurement of the oxygen saturation of retinal vessels in healthy volunteers,” IEEE Trans. Biomed. Eng., 46 (12), 1454 –1465 (1999). https://doi.org/10.1109/10.804573 0018-9294 Google Scholar

33. 

J. M. Beach, K. J. Schwenzer, and S. Srinivas, “Oximetry of retinal vessels by dual-wavelength imaging: calibration and influence of pigmentation,” J. Appl. Physiol., 86 (2), 748 –758 (1999). 8750-7587 Google Scholar

34. 

D. Schweitzer, L. Leistritz, and M. Hammer, “Calibration-free measurement of the oxygen saturation in retinal vessels of men,” Proc. SPIE, 2393 210 –218 (1995). 0277-786X Google Scholar

35. 

D. Schweitzer, M. Hammer, and M. Scribor, “Imaging spectrometry in ophthalmology—principle and applications in microcirculation and in investigation of pigments,” Ophthalmic Res., 28 (suppl 2), 37 –44 (1996). 0030-3747 Google Scholar
©(2007) Society of Photo-Optical Instrumentation Engineers (SPIE)
William R. Johnson, Daniel W. Wilson, Wolfgang Fink, Mark S. Humayun M.D., and Gregory H. Bearman "Snapshot hyperspectral imaging in ophthalmology," Journal of Biomedical Optics 12(1), 014036 (1 January 2007). https://doi.org/10.1117/1.2434950
Published: 1 January 2007
JOURNAL ARTICLE
7 PAGES


SHARE
Advertisement
Advertisement
Back to Top