Open Access Paper
21 June 2019 Automated cell identification with compact field portable 3D optical imaging
Author Affiliations +
Abstract
In this keynote address paper, we overview recently published works on the current techniques and methods for automated cell identification with 3D optical imaging using compact and field portable systems. 3D imaging systems including digital holographic microscopy systems as well as lensless pseudorandom phase encoding systems are capable of capturing 3D information of microscopic objects such as biological cells which allows for highly accurate automated cell identification. Systems based on digital holography enable reconstruction of the cell’s 3D optical path length profile. The reconstructed 3D profiles can be used to extract morphological and spatio-temporal cell features from biological samples for classification and cell identification. Similarly, pseudorandom encoding techniques such as single random phase encoding (SRPE) and double random phase encoding (DRPE) can be used to encode 3D cell information into opto-biological signatures which can be used for cell identification tasks. Recent advancements in these areas are presented including compact and field-portable 3D-printed shearing digital holographic microscopy systems, integration of digital holographic microscopy with head mounted augmented reality devices, and the use of spatio-temporal features extracted from cell membrane fluctuations for sickle cell disease diagnosis.

1.

INTRODUCTION

Compact and field portable three-dimensional (3D) imaging systems such as digital holographic microscopy (DHM) systems and pseudorandom phase encoding systems have become prominent in cell identification tasks because of their capabilities to capture 3D information of biological cells leading to highly accurate automated cell identification [1-19]. In this keynote address paper, we overview previously published works and discuss various configurations, methods, and applications for automated cell identification using compact and field portable 3D optical imaging. This overview includes commonly used holographic interferometric microscopes as well as lensless SRPE and DRPE configurations. Furthermore, recent applications of automated cell identification in 3D optical imaging such as compact and field-portable 3D-printed systems, integration with head mounted augmented reality devices, and the use of dynamic spatio-temporal features derived from cell membrane fluctuations are overviewed.

2.

METHODOLOGY

2.1

Mach-Zehnder microscope

Among the most commonly used microscopes for holographic imaging is the Mach-Zehnder interferometer configuration, as shown in Fig. 1. The Mach-Zehnder microscope employs a beam splitter to split the source laser into an object and reference arm then uses mirrors and a second beam splitter to cause an interference of the two beams at the sensor. In the object arm, the sample and a microscope objective are inserted to capture the object information. In the reference arm, a microscope objective without object is used to match the wavefront curvature produced by the objective in the object arm and provide linear fringes at the hologram plane. Hologram recording is followed by numerical reconstruction [1,2].

Figure 1.

Diagram of the Mach-Zehnder microscope for off-axis digital holographic interferometry [1].

00124_PSISDG11060_1106013_page_2_1.jpg

2.2

Shearing microscope

The shearing microscope is a common-path interferometric microscope which uses the unmodulated portion of the object as a reference to create an interference pattern with improved stability over two-beam systems [1,3]. A schematic of the shearing microscope is shown in Fig. 2 (a). A thick glass plate will reflect the beam from both its front and back surfaces to generate two laterally sheared beams that interfere at the sensor. Figure 2 (b-d) shows recorded red blood cells (RBCs) from a thin blood smear recorded and reconstructed by a shearing microscope (λ = 635 nm, 40X microscope objective, pixel pitch of 4.65 μm).

Figure 2.

a) Schematic of the shearing microscope for off-axis digital holographic interferometry. b) Hologram of red blood cells (RBCs) recorded by the shearing interferometer, c) a zoomed in region for a single RBC showing fringes modulated by the cell, and d) thickness profiles of the RBCs calculated from the extracted phase profiles [1].

00124_PSISDG11060_1106013_page_2_2.jpg

2.3

Lloyd’s mirror wavefront division microscope

Lloyd’s mirror is an alternative method for self-referencing digital holography that uses a mirror to fold a portion of the object beam back onto itself. This approach is advantageous for dealing with the loss of flux caused by a glass plate, and suitable for low coherence length sources. Furthermore, the fringe density can be adjusted by the tilt of the mirror. The optical setup, and reconstruction results for this configuration are shown in Fig. 3. Stability of this system has been reported to be in the sub-nanometer range, making this configuration useful for measuring nanometer-level cell fluctuations and on-site field measurements [1,4].

Figure 3.

a) Diagram of the as Lloyd’s mirror wavefront division, b) hologram of an RBC, c) reconstructed height profile of the RBC [1].

00124_PSISDG11060_1106013_page_3_1.jpg

2.4

Two lens wavefront division microscope

The two lens wavefront division microscope uses a common source that is split by two lenses of equal focal length separated by a small distance, to provide a common-path system with high temporal stability and without a loss in field of view [5]. A schematic for this microscope is presented in figure 4. The sample under consideration is placed under lens L1 and the unperturbed portion of the wavefront passes through lens L2 which generates a reference beam with matched curvature to create an interference pattern at the sensor plane. This configuration has been used to successfully extract cell parameters from RBCs [5].

Figure 4.

Diagram of the two lens wavefront division digital holographic microscope, b) wavefront division module housing the two lenses (L1 and L2), and c) diagram for the conversion of the incident source beam into an object and reference wavefront which interfere at the sensor [5].

00124_PSISDG11060_1106013_page_3_2.jpg

3.

APPLICATIONS

3.1

Lensless cell identification using single and double random phase encoding

In addition to the presented lens-based systems [1-17], lensless cell identification systems [18,19] can quickly and efficiently classify micro-objects without performing 3D reconstruction and are advantageous compared to traditional holography-based systems as they do not have a spatial frequency bandwidth limitation due to the numerical aperture of an objective lens. For lensless cell identification, a compact single-beam system was designed using a laser diode, diffuser, and CMOS sensor to pseudo-randomly encode a target specimen’s complex amplitude, then record its unique opto-biological signature at the sensor plane. Variations in opto-biological signatures due to differences in morphology, size, complex sub-cellular structure, material composition, and index of refraction provide unique signatures for a given specimen can be used for classification. Lensless cell identification system has been presented for single specimen classification using single random phase encoding (SRPE) [18] and both lensless SRPE and double random phase encoding (DRPE) systems for lensless multi-cell identification [19]. Both setups are shown schematically by Figs. 5 (a) and 5 (b) respectively. The experimental setup is shown in Fig 5 (c). The authors note the DRPE system may provide a more robust system as DRPE has been shown to produce a white and wide sense stationary signal. This advantage may make DRPE more robust to translations of the input object.

Figure 5.

a) Single random phase encoding (SRPE) configuration b) double random phase encoding (DRPE) configuration, and c) experimental system used for both SRPE and DRPE experiments [19].

00124_PSISDG11060_1106013_page_4_1.jpg

3.2

Compact and field portable 3D printed shearing DHM for automatic cell identification

Compact and field-portable 3D-printed shearing digital holographic microscopes for automated cell identification have previously been presented [14]. Figure 6 shows two 3D-printed prototypes for portable shearing digital holographic microscopes.

Figure 6.

a) He-Ne based 3D-printed lateral shearing digital holographic microscope, and b) laser diode-based 3D-printed lateral shearing digital holographic microscope [14].

00124_PSISDG11060_1106013_page_4_2.jpg

Reconstruction for a glass bead using this system is shown in Fig 7. (a-e). From the reconstructed profiles, morphological cell features were extracted for classification. The 3D printed system (Fig 6 (b)) was designed such that a regular cell phone camera could be used as the system’s imaging sensor. The He-Ne based system performed best in terms of stability and reported a temporal stability of 0.24 nm as shown by Fig. 7 (f).

Figure 7.

a) hologram of glass bead recorded in the 3D-printed system, b) unwrapped phase profile of the glass bead, c) 2D height map of the glass bead, d) height profile of the glass bead along the line in c, e) 3D reconstruction of the glass bead, and f) temporal stability results for the 3D printed system shown in Fig 6 (a) [14].

00124_PSISDG11060_1106013_page_5_1.jpg

Overall, the reported system was low-cost, stable, and field portable while reporting highly accurate cell identification properties. The system offers potential low-cost alternatives for the developing world or other areas with low access to laboratory facilities.

3.3

Automatic cell identification and visualization with augmented reality devices

Augmented reality smart glasses have also been combined with digital holographic microscopy in a compact and field portable 3D-printed system [15], to allow for quickly visualizing 3D cell profiles along with the cell’s identification and relevant parameters. The optical schematic of the system is shown in figure 8(a) accompanied by two perspective views of the 3D-printed system (Fig. 8 (b) and (c)).

Figure 8.

a) Proposed schematic for the augmented reality digital holographic prototype using a laser source, microscope objective, CMOS image sensor and the augmented reality (AR) device. b) The 3D-printed prototype, and c) the 3D-printed prototype with the AR glasses attached. The Epson Moverio smart glasses were used in the experiments [15].

00124_PSISDG11060_1106013_page_5_2.jpg

Figure 9 shows the five classes of micro-objects used for classification (a-f) and the resulting output as displayed to the augmented reality device (g). The system reported achieved up to 97% classification accuracy among five micro-objects, as well as offering video reconstructions evidenced by a video reconstruction of live paramecium [15].

Figure 9.

(a) Digital hologram of each of the five classes (from left to right: diatom phytoplankton, diatom pennate, navicula, selenastrum, and glass bead), and pseudocolor 3D rendering of the optical path length profile for (b) diatom phytoplankton, (c) diatom pennate, (d) navicula, (e) selenastrum, and (f) glass bead. g) Output as displayed to the augmented reality smart glasses, the source slide is visible in the background [15].

00124_PSISDG11060_1106013_page_6_1.jpg

3.4

Sickle cell disease diagnosis based on spatio-temporal cell dynamics using shearing DHM

Automated diagnosis of sickle cell disease based on red blood cell (RBC) membrane fluctuations measured via digital holographic microscopy was presented using a low-cost, compact, and 3D-printed shearing interferometer [17]. Fig. 10 (a) shows the optical schematic and Fig. 10 (b) shows the 3D-printed prototype, respectively.

Figure 10.

(a) Optical schematic of the overviewed shearing digital holographic microscope for cell analysis. (b) Image of the overviewed 3D-printed compact system [17].

00124_PSISDG11060_1106013_page_6_2.jpg

Example reconstructions for RBCs of healthy and sickle-cell diseased patients are shown by Fig. 11 (a) and (b) respectively.

Figure 11.

3D optical path length profiles of (a) healthy RBC, and (b) sickle cell diseased RBCs [17].

00124_PSISDG11060_1106013_page_7_1.jpg

From the reconstructed frames, a spatio-temporal data cube was formed by stacking the frames sequentially, and features were extracted regarding mean cell fluctuation, and the standard deviation of the cell fluctuations. Additionally, the optical flow algorithm was used to provide a feature measuring lateral cell movement, and 7 morphological features were used in the classification model.

For diagnostic testing, the cells from 2 healthy patients and 2 sickle cell patients were held out for testing while the remaining patients’ cells were used in training. A summary of the classification results is provided in Table 1. Combination of both the spatio-temporal and morphological features proved to be the most successful approach. The use of spatio-temporal features which measure cell membrane fluctuations increased classification performance over morphological features alone.

Table 1:

Summary of classification accuracy outputs for disease diagnosis of patients based on features used [17]

 HealthySCD
Healthy Patient 1Healthy Patient 2SCD Patient 1SCD Patient 2
Spatio-temporal20%30%72%96%
Morphological90%80%92%96%
Combined100%100%100%100%

*Percentages are in terms of how many cells were correctly classified for each patient

4.

CONCLUSION

In summary, we have overviewed previously published works for the methods and strategies of compact and field portable 3D optical imaging for automated cell identification. In particular, we have overviewed the Mach-Zehnder microscope, and several common-path variants for digital holographic microscopy (DHM) including the shearing microscope, the Lloyd’s mirror wavefront division microscope, and the two lens wavefront division microscope. Successful cell identification from opto-biological signatures recorded in single random phase encoding (SRPE) and double random phase encoding (DRPE) arrangements was also presented without requiring cell reconstruction and visualization. Recent advancements and applications in 3D optical imaging for cell identification including field-portable and 3D-printed DHM systems, automatic cell identification and visualization with head mounted augmented reality devices, and sickle cell disease diagnosis based on spatio-temporal cell dynamics were discussed. The various techniques and methods presented in this paper provide several options for highly accurate cell identification through use of compact and field portable 3D optical imaging.

ACKNOWLEDGMENT

T. O’Connor acknowledges support from Department of Education under the GAANN Fellowship. B. Javidi acknowledges support in part by The Office Naval Research under contract N00014-17-1-2405.

REFERENCES

[1] 

Anand, A., Chhaniwal, V., and Javidi, B., “Tutorial: common path self-referencing digital holographic microscopy,” APL Photon., 3 071101 (2018). https://doi.org/10.1063/1.5027081 Google Scholar

[2] 

Schnars, U. and Jueptner, W., “Digital Holography: Digital hologram recording, numerical reconstruction and related techniques,” Springer,2005). Google Scholar

[3] 

Singh, A. S., Anand, A., Leitgeb, R. A., and Javidi, B., “Lateral shearing digital holographic imaging of small biological specimens,” Opt. Express, 20 (21), 23617 –23622 (2012). https://doi.org/10.1364/OE.20.023617 Google Scholar

[4] 

Chhaniwal, V., Singh, A.S.G., Leitgeb, R.A., Javidi, B., and Anand, A., “Quantitative phase-contrast imaging with compact digital holographic microscope employing Lloyd’s mirror,” Opt. Lett., 37 5127 –5129 (2012). https://doi.org/10.1364/OL.37.005127 Google Scholar

[5] 

Patel, N., Trivedi, V., Mahajan, S., Chhaniwal, V., Fournier, C., Lee, S., Javidi, B., and Anand, A., “Wavefront division digital holographic microscopy,” Biomed. Opt. Express, 9 2779 –2784 (2018). https://doi.org/10.1364/BOE.9.002779 Google Scholar

[6] 

Javidi, B., and Tajahuerce, E., “Three-dimensional object recognition by use of digital holography,” Opt. Lett., 25 (9), 610 –612 (2000). https://doi.org/10.1364/OL.25.000610 Google Scholar

[7] 

Javidi, B., Moon, I., Yeom, S. K., and Carapezza, E., “Three-dimensional imaging and recognition of microorganism using single-exposure on-line (SEOL) digital holography,” Opt. Express, 13 (12), 4492 –4506 (2005). https://doi.org/10.1364/OPEX.13.004492 Google Scholar

[8] 

Moon, I., Anand, A., Cruz, M., and Javidi, B., “Identification of Malaria Infected Red Blood Cells via Digital Shearing Interferometry and Statistical Inference,” IEEE Photon. J., 5 (5), 6900207 –6900207 (2013). https://doi.org/10.1109/JPHOT.2013.2278522 Google Scholar

[9] 

Anand, A., I. K. Moon, and Javidi, B., “Automated Disease Identification with 3-D Optical Imaging: A Medical Diagnostic Tool,” in Proc. IEEE, 924 –946 (2017). Google Scholar

[10] 

Shaked, N. T., Satterwhite, L. L., Telen, M. J., Truskey, G. A., and Wax, A., “Quantitative microscopy and nanoscopy of sickle red blood cells performed by wide field digital interferometry,” J. Biomed. Opt., 16 (3), 030506 –030506 (2011). https://doi.org/10.1117/1.3556717 Google Scholar

[11] 

Yi, F., Moon, I., and Javidi, B., “Cell morphology-based classification of red blood cells using holographic imaging informatics,” Biomed. Opt. Express, 7 (6), 2385 –2399 (2016). https://doi.org/10.1364/BOE.7.002385 Google Scholar

[12] 

Matoba, O., Quan, X., Xia, P., Awatsuji, Y., and Nomura, T., “Multimodal Imaging Based on Digital Holography,” in Proc. of IEEE, 906 –923 (2017). Google Scholar

[13] 

Moon, I., and Javidi, B., “Shape tolerant three-dimensional recognition of biological microorganisms using digital holography,” Opt. Express, 13 (23), 9612 –9622 (2005). https://doi.org/10.1364/OPEX.13.009612 Google Scholar

[14] 

Rawat, S., Komatsu, S., Markman, A., Anand, A., and Javidi, B., “Compact and field-portable 3D printed shearing digital holographic microscope for automated cell identification,” Appl. Opt., 56 (9), D127 –D133 (2017). https://doi.org/10.1364/AO.56.00D127 Google Scholar

[15] 

O’Connor, T., Rawat, S., Markman, A., and Javidi, B., “Automatic cell identification and visualization using digital holographic microscopy with head mounted augmented reality devices,” Appl. Opt., 57 B197 –B204 (2018). https://doi.org/10.1364/AO.57.00B197 Google Scholar

[16] 

Anand, A., Chhaniwal, V. K., Patel, N. R., and Javidi, B., “Automatic identification of malaria-infected RBC with digital holographic microscopy using correlation algorithms,” IEEE Photon. J., 4 (5), 1456 –1464 (2012). https://doi.org/10.1109/JPHOT.2012.2210199 Google Scholar

[17] 

Javidi, B., Markman, A., Rawat, S., O’Connor, T., Anand, A., and Andemariam, A., “Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy,” Opt. Express, 26 (10), 13614 –13627 (2018). https://doi.org/10.1364/OE.26.013614 Google Scholar

[18] 

Javidi, B., Rawat, S., Komatsu, S., and Markman, A., “Cell identification using single beam lensless imaging with pseudo-random phase encoding,” Opt. Lett., 41 3663 –3666 (2016). https://doi.org/10.1364/OL.41.003663 Google Scholar

[19] 

Javidi, B., Markman, A., Rawat, S., “Automatic multicell identification using a compact lensless single and double random phase encoding system,” Appl. Opt., 57 B190 –B196 (2018). https://doi.org/10.1364/AO.57.00B190 Google Scholar
© (2019) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Bahram Javidi, Timothy O'Connor, Arun Anand, Inkyu Moon, and Adam Markman "Automated cell identification with compact field portable 3D optical imaging", Proc. SPIE 11060, Optical Methods for Inspection, Characterization, and Imaging of Biomaterials IV, 1106013 (21 June 2019); https://doi.org/10.1117/12.2527573
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Digital holography

Microscopes

3D image processing

Holography

Glasses

Computer programming

Augmented reality

Back to Top