1.

Introduction

Hyperspectral imaging (HSI) is a group of methods for acquisition and analysis of spectrally resolved images.^1–3 In this paper, we consider HSI as a technique only for acquisition of a spectral dataset without regarding further digital postprocessing. Different HSI concepts, such as scanning grating or prism (so-called pushbroom) systems,⁴ tunable monochromatic illumination,^5,6 liquid crystal,^7,8 and acousto-optic tunable filters (AOTFs),^9–11 are used in biomedical optics. HSI systems can be easily integrated with an optical microscope^6,9,11 or with standard endoscopic systems.^5,10 Hyperspectral and multispectral imaging have become stable trends in cancer diagnostics during the last decade.^{5,8,9,12–19}

One of the actual problems in biomedical optics is label-free imaging and analysis of low-contrast objects, such as live cells and tissue cultures, unstained histological sections, and cytological smears.²⁰ Different spatial filtering techniques have originated from Zernike’s phase-contrast imaging.²¹ Traditional phase imaging methods include dark-field, phase-contrast, and differential imaging contrast microscopy. They require special illumination units and additional optical filters of the specimen.²² An alternative approach is to apply a spatial filter or a mask in a Fourier plane in the detector part of the optical system that can be used for imaging of phase objects.^23,24

Our research focuses on the acousto-optic HSI method because of its high spectral and spatial resolution and adaptivity of the angular transfer function.^25–30 An AOTF usually is used as a wide-field tunable monochromator. Ultrasound induces a dynamical phase grating in crystal volume, and diffracted wavelength is determined by the frequency of ultrasound.³¹ Anisotropic Bragg diffraction in birefringent crystals is used in AOTFs. This type of acousto-optic interaction requires linear light polarization at the input and produces orthogonal linear polarization at the output. This is a staring HSI method with fast random-access wavelength tuning, i.e., during each exposure of a two-dimensional (2-D) detector array, the whole scene is captured, and the recorded wavelength can arbitrarily vary from one exposure to another. Typical switching time of an AOTF from one wavelength to another is about $10\mu \mathrm{s}$. Thus, a full HSI dataset acquisition can be as short as a few seconds, which is important for operational diagnostics and real-time surgery assistance.^5,8 Recent developments in digital methods of arbitrary ultrasonic field synthesis can be useful for advanced spectral processing by means of AOTFs.^32,33 It is also known that image edge enhancement can be effectively performed by AOTFs with coherent object illumination.^{11,25–28,34–36}

Using the peculiarities of the AOTF transfer function for visualization of phase objects has been proposed and studied by Balakshy et al.^26–28,34 It has been shown that, unlike the dark-field and the phase-contrast methods, acousto-optic filtering results in intensity at the image plane proportional to the optical phase gradient in the object. The disadvantage of this method is that it works with coherent light only. Applying it to HSI would require a tunable monochromatic illumination unit resulting in modifications of a standard microscope. Moreover, such an approach is not applicable to hyperspectral fluorescence imaging. Other acousto-optic phase imaging techniques include interferometry, which requires a modification of a microscope.³⁷

In this paper, we experimentally demonstrate simultaneous spatial and spectral filtering of images obtained with incoherent light for the purpose of imaging unstained low-contrast objects. We developed an original acousto-optic HSI system that performs in a phase gradient visualization mode of image processing in addition to the conventional tunable selection of transmitted wavelength. The new feature has become available using a special amplitude mask (AM) in the Fourier plane for spatial filtering and an AOTF as a tunable monochromator. A telecentric optical system was used to match transmission of the mask with the transfer function of the AOTF. Our HSI system operates with a commercial bright-field inverted microscope. The system can be used in two modes: (1) conventional HSI data acquisition and (2) bandpass spatial filtering of narrowband diffracted light. The performance of the imaging system is demonstrated with unstained biopsies and histological sections of surgically removed tissue.

2.

Method

One of the common optical schemes of AOTF-based HSI systems is a confocal one.^38,39 In this layout, the field is imaged at the AOTF aperture, and the diffracted beam is relayed to the detector array by the secondary optical system. The confocal system provides low aberrations except for longitudinal chromatism. The latter can be compensated for using an apochromatic design of the relay optical system.²⁹ An important property of a confocal system is that the AOTF is placed at the intermediate image plane. The angular spectrum at the detector is a product of the incoming beam spectrum, the transfer function of the optical system, and the AOTF transfer function.²¹

The proposed optical system used for image filtering in our experiments is shown in Fig. 1. The original beam is a broadband light focused by the microscope at the input plane of the HSI system, $z=0$. The field lens (FL) corrects the beam path so that the zero spatial frequency of the input beam crosses the optical axis of the system at the front focal plane of the first relay lens (${\mathrm{RL}}_1$), $z=f_1$. An AM is placed there to suppress low spatial frequencies, i.e., for spatial filtering of the beam. The beam is focused onto the AOTF, and the spatial frequency $k$ at $z=4f_1$ is related to the distance from the axis in the mask plane, $r$, as

Fig 1

Simplified optical scheme of a confocal telecentric acousto-optic HSI system with spatial filtering. FL, field lens; ${\mathrm{RL}}_1$ and ${\mathrm{RL}}_2$, relay lenses; and hatching indicating the spatial frequencies transmitted by the mask.

In HSI applications, the noncritical phase matching (NPM) condition in noncollinear AOTFs is usually used.⁴⁰ Analysis of the AOTF transfer function is based on the concepts of Fourier optics and geometrical consideration of phase mismatch $R$. Hereinafter, $R$ is a dimensionless product of wave vector difference by interaction length. As follows from the coupled modes equations, the AOTF transfer function $H_{\mathrm{ao}}(k_x,k_y)$ at the optimum coupling coefficient can be expressed as³¹

In addition to the spatial frequencies $k_x$ and $k_y$, the phase mismatch $R$ depends on the optical wavelength $\lambda$ and the frequency of ultrasound. The maximum of the transfer function is obtained at zero phase mismatch, and $|R|=0.8$ is the criterion for $-3\mathrm{dB}$ diffraction efficiency drop. The phase mismatch in NPM diffraction geometry is a quadratic function of angular frequencies that provides a wide acceptance angle. A typical numerical aperture of a paratellurite AOTF is $1/10$. This feature is used to obtain a high spatial resolution in conventional HSI applications of AOTF. Strictly speaking, an AOTF is not an axially symmetrical optical element; therefore, it can cause astigmatism and angular dispersion.³⁰ However, the angular dispersion is cancelled in the confocal scheme,³⁸ and astigmatism is minimized if the diffracted beam is parallel to the input one.

The phase mismatch and the AOTF transfer function are plotted in Fig. 2. Here, we consider some fixed wavelength $\lambda$ within the AOTF tuning range. The symmetry of the transfer function is close to circular for typical configurations of AOTFs.^11,25,26,28 For this reason, in Fig. 2, only one section of the transfer function along the $y$-axis (orthogonal to the diffraction plane) is shown. When the frequency of ultrasound is exactly matched at the zero spatial frequency, the NPM condition is satisfied. While increasing the frequency of ultrasound, the mismatch curve is shifted toward negative values, and phase matching occurs at higher spatial frequencies. The loci of phase matching points in the $(k_x,k_y)$ domain are an oval for a detuned NPM configuration.^11,26 To match the input angular spectrum at the AOTF input with a ring-shaped transfer function, we have chosen an AM transmitting light in the area

Fig 2

NPM in an AOTF and spatial filtering: (a) phase mismatch, (b) AOTF transfer function, and (c) input angular spectrum filtering by the telecentric AM.

The calculation results presented in Fig. 2 were obtained for the wavelength $\lambda =633\mathrm{nm}$. The NPM condition takes place at the ultrasonic frequency of 132.8 MHz; detuned NPM geometry is at the frequency of 133.1 MHz. The cutoff spatial frequencies $k_{\min }$ and $k_{\max }$ correspond to the mask inner and outer radii of 1.5 and 3.0 mm, respectively, provided the focal length is $f_1=100\mathrm{mm}$.

In contrast to previously reported acousto-optic HSI systems,^{10,11,29,30,38,39,41} our schemes are based on a spatial filter and a single AOTF as a monochromator in the detection part of the optical scheme. Lower spatial frequencies at $|k|<k_{\min }$ are blocked by the mask, resulting in suppression of uniform background in the image. However, for a fixed mask configuration, the cutoff angular frequencies $k_{\min }$ and $k_{\max }$ are inversely proportional to the wavelength $\lambda$ according to Eq. (3). In the AOTF, only a narrow spectral bandwidth $\delta \lambda$ is diffracted according to the phase matching condition. As a result, the angular spectrum of filtered light at the CCD contains only those spatial frequencies that lay between the cutoff frequencies for the diffracted wavelength bandwidth.

A peculiar angular spectrum of input beam contains higher spatial frequencies that originate from phase and/or amplitude variations of the specimen transmission. Previously reported methods of gradient phase visualization are based on adjusting the angular spectrum of incoming light to the slope of the transfer function.^{23,24,27,28,34} This results in directional highlighting of phase gradients in the images. Since we use a symmetrical ring-shaped transfer function of the optical system, the processed images are characterized by nondirectional phase visualization.

3.

Optical System Design

A custom confocal acousto-optic HSI system for an infinity-conjugated microscope has been designed and built in-house. The layout of the experimental setup is shown in Fig. 3. An inverted optical microscope (Nikon, Ti-E) was used. The specimen was illuminated with a standard diascopic bright-field condenser. Reference bright-field images were captured with a color CCD camera (Nikon, DS-Fi2) at the eyepiece side port. The AM was located at the front focal plane of the first relay lens (${\mathrm{RL}}_1$, $f_1=100\mathrm{mm}$). Since the optical scheme of the microscope is not telecentric, we used an FL with a shorter focal length, $f_{\mathrm{fl}}\approx 96\mathrm{mm}$. It was enough for proper correction of the beam path with $10\times$ to $40\times$ microscope objectives (Nikon, Plan Apo $\lambda$). In proof-of-principle experiments, we used manually replaceable circular beam stops centered at the beam axis, but instead a 2-D liquid crystal spatial light modulator can be used for automated system operation. Three folding mirrors (${\mathrm{M}}_1$, ${\mathrm{M}}_2$, and ${\mathrm{M}}_3$) were used for wrapping the optical system. Two of them also provided accurate beam alignment by means of pitch and yaw adjustments. Crossed polarizers (${\mathrm{P}}_1$ and ${\mathrm{P}}_2$) provided the vertical polarization at the input of the AOTF and transmitted horizontally polarized diffracted light at the AOTF output. The custom-built AOTF operating in visible and near-infrared with the spectral passband $\delta \lambda /{\lambda }^2=20{\mathrm{cm}}^{-1}$ has been designed and described previously.^11,41 The diffraction plane was horizontal (parallel to the breadboard). The second relay lens (${\mathrm{RL}}_2$, $f_2=50\mathrm{mm}$) imaged the diffracted field at the CCD matrix (Apogee, Alta U32, $6.8\mu \mathrm{m}/\text{pixel}$). The output iris diaphragm (D) was used to stop the residual stray light from the zero diffraction order.

Fig 3

(a) Schematic and (b) photograph of the experimental setup. FL, field lens; AM, amplitude mask; ${\mathrm{M}}_1$, ${\mathrm{M}}_2$, and ${\mathrm{M}}_3$, folding plane mirrors; ${\mathrm{RL}}_1$ and ${\mathrm{RL}}_2$, relay lenses; ${\mathrm{P}}_1$ and ${\mathrm{P}}_2$, polarizers; D, output diaphragm; CCD, CCD camera; and arrows show the beam path.

4.

Validation Experiments

In the experiments, we compared the performance of the proposed bandpass spatial filtering with the standard HSI technique performed with the same experimental setup without the AM. The captured images were not modified digitally except for intensity level adjustments not affecting the contrast.

An unstained cytological smear with human red blood cells from a biopsy was used as a test object (Fig. 4). The plots show a reference color image (a), a standard spectral image (b), and a spectral image after spatial filtering (c). Both images (b) and (c) were obtained at the wavelength $\lambda =660\mathrm{nm}$. An intensity profile of the image fragment with a section of a single cell is shown in Fig. 4(d). The horizontal section was centered on a cell marked with an arrow. Both the conventional bright-field image and the standard spectral image show the dark contours of single cells, while the spatial filtering operation mode suppresses the background and highlights the inner structure of the cells. The contrast between the cell and the background in spatial filtering mode is $\sim 50\%$ higher than in HSI mode.

Fig 4

Micrographs of unstained human red blood cells in a biopsy cytological smear: (a) wide-field image, RGB; (b) conventional spectral image, grayscale; (c) spectral image with contrast enhancement, grayscale; and (d) image intensity profiles through a single cell (marked with an arrow in plots a, b, and c). Transmitted wavelength $\lambda =660\mathrm{nm}$, passband $\delta \lambda =0.9\mathrm{nm}$, and objective magnification $20\times$.

The scene is usually more complicated when the object is a histological section. A heterogenous tissue structure can contain different types of cells and stromal components, e.g., collagen fibers. A specimen in Fig. 5 is an unstained histological section of a surgically removed human malignant tumor, namely, papillary thyroid carcinoma. The same fragment of the specimen was captured at two different objective magnifications: $20\times$ and $40\times$. The diffracted wavelength was $\lambda =610\mathrm{nm}$.

Fig 5

Micrographs of a thyroid histological section: (a) and (c) conventional spectral image and (b) and (d) spectral image with bandpass spatial filtering. Transmitted wavelength $\lambda =610\mathrm{nm}$ and passband $\delta \lambda =0.75\mathrm{nm}$. Objective magnification: (a and b) $20\times$; (c and d) $40\times$. Marked square area in (a) and (b) corresponds to the field of view in (c) and (d), respectively.

Imaging of stained specimens demonstrates the performance of acousto-optic spatial filtering when the object transmission spatially varies both in phase and in amplitude. We used a thyroid histological section stained with haematoxylin and eosin (H&E). Spectral filtering was performed at two different wavelengths: 660 and 810 nm. The results are shown in Fig. 6. At the wavelength of 660 nm, the stained tissue structures are not transparent and are clearly distinguishable in the spectral image (a). The spatially filtered image (b) demonstrates the effect of simultaneous amplitude (darker cell nuclei) and phase (highlighted contours) transmission variations in the specimen. In near-infrared, $\lambda =810\mathrm{nm}$, all structures in the specimen become transparent (c), but the phase modulation of transmitted light remains and can be seen after spatial filtering (d).

Fig 6

Micrographs of an H&E-stained thyroid histological section: (a) and (c) conventional spectral image and (b) and (d) spectral image with bandpass spatial filtering. Transmitted wavelength and passband are wavelength (a and b) $\lambda =660\mathrm{nm}$, $\delta \lambda =0.9$; (c and d) $\lambda =810\mathrm{nm}$, $\delta \lambda =1.3\mathrm{nm}$. Objective magnification $40\times$.

Summarizing the experimental results, we can mention that the proposed method of spatial filtering provides a new modality of spectral imaging. The suppression of uniform background by blocking optical spatial frequencies around zero enables phase visualization. This image processing regime can be used independently or together with conventional HSI providing additional information for data postprocessing. The HSI system retains high spectral resolution of the AOTF since spatial filtering is matched with the AOTF transfer function.

5.

Summary

An optical phase visualization method for advanced hyperspectral image processing has been demonstrated in a custom-built acousto-optic HSI instrument with a commercial bright-field microscope using broadband diascopic incoherent illumination. For this purpose, an additional AM providing spatial filtering in a telecentric confocal system was used. The angular spectrum of light transmitted by the mask was matched with the transfer function of the AOTF in a detuned NPM geometry. Without using the mask, the system was operating in a conventional hyperspectal image processing mode. The benefit of the system is high spectral resolution of HSI data acquisition in real time owing to using an AOTF as a monochromator.

All image processing in our system is performed in the detector part of the optical system. The processing of incoherent light with a standard microscope by the proposed system allows using unmodified diascopic illumination for transmission imaging. The method can also be applied without constraints for processing of epifluorescence images. Higher contrast and phase visualization was demonstrated in bandpass spatial filtering mode using unstained cytological smears of human red blood cells and histological sections of human thyroid as test specimens. The proposed method of image processing can be useful for visualization of low-contrast objects in microbiological and medical HSI for research and diagnostics.