2.2.1.

Discrete Fresnel transformation

In the Fresnel approximation, the factor $\rho$ is replaced by the distance $d^{\prime }$ in the denominator of Eq. (1) and the square root in the argument of the exponential function is replaced by the first terms of a binomial expansion. When terms of higher order than the first two are excluded, $\rho$ becomes

Since $\rho$ appear in the exponent, neglecting higher-order terms than first one, represents very small phase errors. A sufficient condition¹³ is that the distance $d^{\prime }$ is large enough.

Since this is an overly stringent condition, even shorter distances produce accurate results. Since the exponent is the most critical factor, dropping all terms but the first in the denominator produces only acceptable errors only. Thus, the propagation integral in Eq. (1) becomes

The discrete finite form of Eq. (7) is obtained through the pixel size $(\mathrm{\Delta }\xi ,\mathrm{\Delta }\eta )$ of the CCD array, which is different from that $(\mathrm{\Delta }{x}^{\prime },\mathrm{\Delta }{y}^{\prime })$ in the image plane $x^{\prime }-y^{\prime }$ and is related as follows:

According to Eq. (7), the wave field $b^{\prime }(x^{\prime },y^{\prime })$ is essentially determined by the two-dimensional (2-D) Fourier transform of the quantity $h(\xi ,\eta )r(\xi ,\eta )g(\xi ,\eta )$. For rapid numerical calculations, a discrete formulation of Eq. (4) involving a 2-D FFT algorithm is used, as shown in

In the formulation based on Eq. (10), the reconstructed image is enlarged or contracted according to the depth $d^{\prime }$, see Eq. (9).

2.2.2.

Reconstruction by the convolution approach

This is an alternative approach, useful for keeping the size of the reconstructed image constant.¹⁵ In this formulation, the wave field $b^{\prime }(x^{\prime },y^{\prime },d^{\prime })$ can be calculated by

Equation (12) shows that the linear system characterized by $f(\xi ,\eta ,x^{\prime },y^{\prime })=f(\xi -x^{\prime },\eta -y^{\prime })$ is space-invariant: the integral in Eq. (16) is a convolution. This allows the application of the convolution theorem;¹³ thus, the wave field can be found as the inverse transform.

With this method, the size of the reconstructed image does not change in respect to the hologram plane $\mathrm{\Delta }{x}^{\prime }=\mathrm{\Delta }\xi$, $\mathrm{\Delta }{y}^{\prime }=\mathrm{\Delta }\eta$ and it is necessary to have one Fourier transform and one inverse Fourier transform to obtain one 2-D reconstructed image at a distance $d^{\prime }$. Indeed, an analytical version of $F\{f\}$ is readily available, saving one Fourier transform in Eq. (13).

Although the computational procedure is heavier in this case compared to the Fresnel approximation approach of Eq. (10), this method allows for easy comparison of the reconstructed images at different distances $d^{\prime }$, since the size does not change with modifying the reconstruction distance. Furthermore, in this case, we get an exact solution to the diffraction integral as long as the sampling Nyquist theorem is not violated.

2.2.3.

Angular spectrum method

Another possible solution is to identify the complex field as a composition of plane waves traveling in different directions away from a plane.¹⁶ The propagated field across any other parallel plane can be calculated by adding the contributions of these plane waves, with different phase delays, depending on the plane wave’s angle of propagation.

In other words, if the angular spectrum is defined as the Fourier transform of the complex wave field at plane $z=0$

This planes-waves decomposition approach presents many attractive features: it does not require any Rayleigh-Sommerfeld diffraction integral approximations, and, in this case, fast numerical implementations can be used.

3.1.1.

Sectioning and merging approach

In holographic microscopy, the EFI concept has been extended by Ferraro et al.,²⁰ who refer to the merged image from differently focused subareas as the extended focus image. They used the distance information, carried by the phase image, for correct selection of the in-focus portions that have to be selected from each image stack. This will result in the correct construction of the final EFI, provided that some solutions are adopted to control the size of the object independent of the reconstruction distance, and centering it by appropriately modeling the reference beam.²¹

In practice, they noted that phase map $\psi (x^{\prime },y^{\prime })$ in DH incorporates information about the topographic profile of the object under investigation. In fact, the optical path difference (OPD) is related to the phase map by the following equation:

If $p$ is the distance from the object lower point to the lens and $q$ is the corresponding distance on the image plane, then any other point of the object at a different height $\mathrm{\Delta }p(x^{\prime },y^{\prime })$ results in a good focus at different imaging planes in front of the CCD according to the following simple relation:

In a reflection configuration $\mathrm{OPD}(x^{\prime },y^{\prime })=2\mathrm{\Delta }p(x^{\prime },y^{\prime })$, and taking into account Eqs. (17) and (18), they obtained the range of distances at which the digital hologram has to be reconstructed to image all the volume in focus:

Figure 2 represents the conceptual flow process to get the EFI from a digital hologram of a micro-electromechanical system (MEMS):

Fig. 2

Conceptual flow chart describing how the extended focused image (EFI) is obtained by digital holography approach. Images are from Ref. 20.

Later, in addition to reflection configuration, Colomb et al.²² extended this scheme to transmission one. Furthermore, they generalized the application to other areas, such as metrology. For example, the method is employed on phase reconstructions of micro-optics (microlens and retroreflector, see Fig. 3), as well as on amplitude ones. They extracted the extended focus phase image from a stack of $N$ reconstructions using a generalized reconstruction distance map.

Fig. 3

Amplitude (1) and phase (2) reconstructions of a high aspect-ratio retroreflector immersed in distilled water measured in transmission for different reconstruction distances (a) 3.6 cm, (b) 6.6 cm, (c) 11.0 cm, and (d) EFI. Images are from Ref. 22.

Figure 3 presents the amplitude and phase reconstructions obtained for a high-aspect-ratio retroreflector, measured in transmission at $\lambda =664\mathrm{nm}$ and computed with different reconstruction distances. The reconstruction distance map is computed by adjusting the reconstruction distance $d_0=3.6\mathrm{cm}$ to focus the retroreflector edges [Fig. 3(a)]. The EFI for the amplitude and phase are presented in Fig. 3(d). Ultimately, this method allows reconstruction of not only the extended focused amplitude images, but also, especially, the real topography for an object higher than the DOF of the microscope objective.

A typical drawback of this digital holographic EFI technology is that it works only with a single object with an axial dimension larger than the DOF.²³ Instead, in case of multiple objects sparsely distributed in the space, or when the 3-D object shape is not continuous or slowly varies, such as step-like height structures, it has difficulty in automatically identifying multiple, unknown shaped targets and transferring them into their respective best focal position.

In this case, an algorithm able to recognize the presence of multiple targets should be used. It provides a chance to deal separately with these objects, and for each one, a map of heights is to be calculated. At this point, it refocuses each target, respectively, to their best focal planes and, finally, merges them back to form a high-precision 3-D shape result. Also, this type of technique belongs to the category of so-called sectioning and merging, and several attempts have been presented.

For example, by the independent component analysis technique or discrete wavelet transform, Do et al.^24–26 have synthetized an EFI from reconstructed holographic images of many 3-D objects at different in-focus distances. They achieved visual success. Nevertheless, their methods incurred the problem of blurring, since in the merge phase, more or less out-of-focus images are taken into account.

For optical scanning holography, some authors^27–29 have suggested modeling the task of sectioning as an inverse problem and Wiener filtering or iterative algorithms were implemented. Although these methods have reported remarkable results, they only worked for amplitude recovery. Holographic phase information was lost during processing so they cannot be used for purely phase objects.

A most effective approach is to separate the whole image into small blocks, as described in Refs. 3031. to 32. A focal measurement algorithm is applied to each individual block and the best focal position is calculated. EFI is sewn by taking the best focal positions for all blocks. When a large number of objects are present, such as small particles, this idea can be brought to the limit, assessing the best distance pixel-wise to obtain the depth map for each pixel of the image.³³

Anyway, a possible critical point is the choice of focus detection criteria.

Typically, many reconstructed frames are collected along the axial direction, and the best focus plane is chosen by a certain kind of sharpness indicator. A number of various focus metrics have been proposed using an intensity gradient,³⁴ self-entropy,^35,36 gray-level variance,³⁷ spectral $l_1$ norms,³⁸ wavelet theory,³⁹ and stereo disparity,⁴⁰ among others; for a comparison between these methods, see Ref. 41. The majority of focus-finding applications consist of looking for the amplitude extrema, even though in many cases it is phase contrast that is actually of interest. However, another problem arises when, in the examined block, there are not enough features (either presenting no object or being occupied by a whole object yet with no significant change during digital refocusing), which makes it difficult to find the exact focus plane with the focus detection algorithms.

3.1.2.

Multiplane imaging

In many fields of science, such as imaging particle fields, in vivo microscopy, optical propagation studies, wavefront sensing, or medical imaging, multiplane imaging is very common and useful, allowing simultaneous visualization of several layers within the imaged volume.⁴² This is another way to preserve a wide DOF without sacrificing the axial resolution of the objective lens. In practice, the imaging path is multiplexed with beam splitters into multiple paths, each with a different focal length and its own camera for imaging.⁴³ In this way, full axial resolution of the microscope objective is maintained in each of the recorded images. Nevertheless, this approach is quite impractical and has different limitations.

A different and smart approach has been proposed in the work of Blanchard and Greenaway⁴⁴ in which a diffraction grating has been adopted in the optical setup to split the propagating optical field into three diffraction orders (i.e., 1, 0, $+1$). The grating was distorted with an opportune quadratic deformation and, consequently, the wave field resulting from each diffraction order could form an image of a different object plane. A further investigation was published some years later, in which the focusing properties of a diffraction grating having parabolic grooves has been exploited for extending the depth of focus.^45,46 More recently, remarkable progress has been made in the use of a quadratic deformed grating for multiplane imaging of biological samples to demonstrate nanoparticle tracking with nanometer resolution along the optical axis.⁴⁷

To confirm the high interest in multiplexing imaging, in Ref. 48, an approach named depth of field multiplexing is reported. A high-resolution spatial light modulator was adopted in a standard microscope to generate a set of superposed multifocal off-axis Fresnel lenses, which sharply image different focal planes. This approach provides simultaneous imaging of different focal planes in a sample using only a single camera exposure. The maximum number of imaged axial planes is further increased in Ref. 49 using colored RGB illumination and detection. In their paper, the authors have demonstrated the synchronous imaging of as many as 21 different planes in a single snapshot under certain conditions.

In DH, Paturzo and Finizio⁵⁰ demonstrated that the synthetic diffraction grating can be included in the numerical reconstruction to simultaneously image three planes at different depths.

In practice, in the numerical reconstruction algorithm, the hologram is multiplied by the transmission function of a quadratically distorted grating.

The insertion of such a digital grating allows the simultaneous imaging of three object planes at different depths in the same field of view. In fact, the digital deformed grating has a focusing power in the nonzero orders and, therefore, acts as a set of three lens of positive, neutral, and negative powers. In the reconstruction plane, three replicas of the image appear; each one is associated with a diffraction order and has a different level of defocus. The distance from the object plane, corresponding to the $i$’th order, to that in the zeroth order is given by

To demonstrate their technique, they performed different experiments. In the first case, three different wires were positioned at different distances from the CCD array of 100, 125, and 150 mm, respectively. A digital hologram was recorded in a lens-less configuration. They performed two numerical reconstructions of the corresponding hologram at 125 mm, the in-focus distance of the twisted wire, but with two different quadratic deformations of the numerical grating, that is two different values of the parameter $A$. Figure 4 shows the amplitudes of the obtained reconstructions.

Fig. 4

Numerical reconstructions of the “three wires” hologram at $d=125\mathrm{mm}$, the in-focus distance of the twisted wire, with two different values of the numerical grating quadratic deformation in order to obtain: (a) the vertical wire in focus in the $-1$ order image and (b) the horizontal wire in focus in the $+1$ order image. Images are from Ref. 50.

As a further experiment they also applied the method to holograms of a biological sample. The specimen is formed by three in vitro mouse preadipocyte 3T3-F442A cells that are at different depths. Figure 5 shows the amplitude reconstruction at a distance $d=105\mathrm{mm}$ at which the cell indicated by the blue arrow is in focus (see the zeroth-order image). The $+1$ order corresponds to a distance $d=92.7\mathrm{mm}$ at which the cell indicated by the yellow arrow is in good focus, while the $-1$ order corresponds to a depth of $d=121\mathrm{mm}$, where the filaments are visible (highlighted by the red ellipse in Fig. 5).

Fig. 5

Amplitude reconstruction of a “cells” hologram at a distance $d=105\mathrm{mm}$ at which the cell indicated by the blue arrow is in focus. The $+1$ order corresponds to a distance $d=92.7\mathrm{mm}$ at which the cell indicated by the yellow arrow is in good focus, while the $-1$ order corresponds to a depth of $d=121\mathrm{mm}$ where the filaments are well visible, highlighted by the red ellipse. Images are from Ref. 50.

The use of a numerical grating instead of a physical one, has the great advantage of increasing the flexibility of the system. For example, depending on the grating period and the amount of deformation, the distance of the multiple planes can be easily changed and adapted to the needs of the observer.

Moreover, they verified that the adoption of a deformed diffraction grating can be exploited in multiwavelength DH.

Afterward, Pan⁵¹ presents an angular spectrum method (ASM)-based reconstruction algorithm to simultaneously image multiple planes at different depths. A shift parameter is introduced in the diffraction integral kernel. It takes account of the coordinate system’s transverse displacement of the image plane at different depths. A combination of the diffraction integral kernel with different shift values and reconstruction depths yields multiplane imaging resolution in a single reconstruction. Furthermore, a method to extend the depth of focus using a single-shot digital hologram is also proposed.

3.1.3.

3-D imaging

The very important advantage of DH is that all the 3-D information intrinsically contained in the digital hologram, can be usefully employed to construct a single image with all portions of a 3-D object in good focus. However, the question of the relationship between the 3-D distribution of the wave field and the configuration of the object is still not solved.

Consider the first case with a single wavelength and a single propagation direction of the illuminating wave (single $k$-vector): the reconstructed wavefront contains all contributions originating from all parts of the specimen and cannot be considered as the true 3-D image of the object. Indeed, the coherent source produces interferences with each of the reflected or transmitted waves or, more generally, diffracted waves coming from each part of the object.

The final image is then the superposition of the contributions from all the sections, in addition to the one where the wavefront is reconstructed (in-focus plan). The contributions of the upper and lower sections of the object (out-of-focus plans), therefore, appear as undesired contributions that blur the image. A major objective of the research is to adequately solve the problem of true 3-D object imaging by the elimination of all unwanted contributions.

In holographic microscopy, different strategies exist to solve this problem.

Initially, Onural⁵² extended the impulse function concept over a curve or a surface and he used it to improve the structure of the diffraction problem formulation, thus paving the way for elegant solutions of many associated problems. However, these require 3-D Fourier transforms, integrals over surface, and rotation matrices, making the problem numerically difficult to treat.

In Refs. 53 and 54, 3-D deconvolution methods with a point spread function (PSF) are extended to holographic reconstructions with the aim to rebuild the true 3-D distribution of small particles. Unfortunately, 3-D deconvolution products require a high amount of memory and data resampling is often necessary, implying a loss of spatial resolution.

3-D data were retrieved by Pégard and Fleischer⁵⁵ using 3-D deconvolution in microfluidic microscopy. In particular, the focal stack generated by tracking samples flowing into a tilted microfluidic channel [see Fig. 6(a)] and the system PSF [Fig. 6(b)] are processed in a Wiener deconvolution filter to extract size, position, orientation, and subcellular surface features of aggregated yeast cells, Fig. 6(c).

Fig. 6

(a) Focal stack and (b) point spread function (PSF) focal stacks are recorded in a deconvolution microfluidic microscopy. The three-dimensional (3-D) structure of the object is deconvolved and an iso-level surface showing the 3-D envelope of yeast particles is displayed (c). Images are from Ref. 55.

In diffractive tomography, Cotte at al.⁵⁶ combined the theory of coherent image formation and diffraction. Through an inverse filtering obtained by a realistic coherent transfer function, namely 3-D complex deconvolution, they enabled the reconstruction of an object scattered field. The authors expected this technique to lead to aberration correction and improved resolution.

By combining the advantages of full-field frequency-domain optical coherence tomography with those of photorefractive holography, Koukourakis et al.⁵⁷ proposed a system for a complete 3-D image. In their system, a 3-D stack of spectral interferograms is constructed to obtain depth information. This setup employs a wavelength scanning tunable laser as the light source, and the use of a photorefractive medium to holographically store the spectral interferograms obtained by scanning the wavelength.

3.2.1.

Diffraction between arbitrarily oriented planes

Leseberg and Frère⁵⁸ were the first who addressed the problem of describing the diffraction pattern of a tilted plane using Fresnel approximation. It is calculated by a Fourier transformation, a coordinate transformation, and a multiplication by a quadratic phase.

Later on, a general-purpose numerical method for analyzing optical systems by the use of full scalar diffraction theory was proposed by Delen.⁵⁹ His approach is based on Rayleigh-Sommerfeld diffraction and it can be applied to wide angle diffraction. In particular, the author proposed two methods, one for shifted plane and the other one for tilted plane, and these can be sequentially combined for shifted and tilted planes. This is a very advantageous feature, because other methods are limited to rotation around one axis. For example, Yu et al.⁶⁰ used Fourier transform method (FTM) for numerical reconstruction of digital holograms with changing viewing angles.

Certainly, the use of the a plane waves angular spectrum and coordinates rotations represents a more flexible solution. Initially, Tommasi and Bianco⁶¹ proposed a technique for finding the relation between the plane-wave spectra of the same field, with respect to two coordinate systems rotated only with respect to each other, to calculate the computer-generated holograms of off-axis objects. Subsequently, De Nicola et al.⁶² and Matsushima⁶³ proposed methods to obtain the EFI of objects or the target recorded on inclined planes, by taking the angular spectrum into consideration.

The angular spectrum-based algorithm for reconstructing the wave field on arbitrary inclined plane basically consists of two steps. In the first one, the angular spectrum $A(u,v;d^{\prime })$ is calculated on an intermediate plane ($x\text{-}y$) at distance $d^{\prime }$. Standard transformation matrix is then used to rotate the wave vector coordinates. This matrix is, in general, given as a rotation matrix $R_y({\theta }_y)$ or the product of several rotation matrices.

In the second step, the rotate spectrum is inverse Fourier transformed to calculate the reconstructed wave field on the tilted plane, namely

It should be remarked that reconstructing the field according to Eq. (24) is valid within the paraxial approximation. However, it can be generalized to include frequency-dependent terms of the Jacobian associated to rotation. Furthermore, the spectrum should be shifted in the reference Fourier space. Since the complex value of the spectrum has to be obtained for each sampling point on the equidistant sampling grid, interpolation is needed because of the nonlinearity attributed to Eq. (24).

In summary, in these cases, fast Fourier transformation is used twice, and coordinate rotation of the spectrum enables one to reconstruct the hologram on the tilted plane. Interpolation of the spectral data is shown to be effective for correcting the anamorphism of the reconstructed image.

In Fig. 8, a case of two-axis rotation is shown.⁶³ The planar object is slanted at 30 deg around the y axis after rotation at $-60\mathrm{deg}$ around the $x$ axis. Therefore, the transformation matrix $T=R_y(30\mathrm{deg})R_x(-60\mathrm{deg})$ is used to retrieve the original pattern.

Fig. 8

Amplitude images (a) reconstructed in the parallel plane and (b) in the tilted plane reconstructed by using rotational transformation for two-axis rotation. The planar object is rotated at $-60\mathrm{deg}$ around the $x$ axis prior to rotation at 30 deg around the $y$ axis. Images are from Ref. 63.

Since these methods suffer from the loss of resolution problem, Jeong and Hong⁶⁴ have presented an effective method for the pixel-size-maintained reconstruction of images on arbitrarily tilted planes. The method is based on the plane wave expansion of the diffraction wave fields and on the three-axis rotation of the wave vectors. The images on the tilted planes are reconstructed without loss of the frequency contents of the hologram and have the same pixel sizes. For example, Fig. 9(a) presents the hologram reconstruction of a 1951 USAF target rotated by ${\theta }_y=-45\mathrm{deg}$ and ${\theta }_x=40\mathrm{deg}$ on the plane parallel to the CCD plane at $z=-2.20\mathrm{cm}$ using the ASM. The resolution target’s center is located on the optic $z$ axis at 2.42 cm in front of the CCD plane. It can be seen that the left-hand upper corner of the image is focused, while the other parts are out of focus because of object tilting. Figure 9(b) is the image at $z=-2.42\mathrm{cm}$ reconstructed with a correction method; it is focused across the whole area of the resolution target, but its pixel size is $\simeq 0.7\text{times}$ smaller than that of the hologram due to the scaling caused by the FFT. The image in Fig. 9(d), which was reconstructed by their method, is focused across the whole plane, and the ratio between the $x$ and the $y$ dimensions of the reconstructed resolution target is the same as that of the real object, which proves that their method can faithfully reconstruct images on the tilted planes.

Fig. 9

Reconstructed images of the tilted resolution target ($640\times 480\text{pixels}$). (a) Image on the plane parallel to the CCD at $z=-2.20\mathrm{cm}$ reconstructed with the ASM. (b) Images on the tilted plane at $z=-2.42\mathrm{cm}$ reconstructed from the whole area. (c) Image reconstructed by the method in Ref. 64.

The popularity of these techniques is now so extended that they are also successfully applied in many others fields such as biology.^65–67 In particular, in the paradigm of TIR holographic microscopy, Ash et al. used angular spectrum rotation for imaging organisms, cell-substrate interfaces, adhesions, and tissue structures. Figure 10 shows a basic configuration of the interferometer for digital holographic microscopy of TIR. The object beam enters the prism and undergoes TIR at the hypotenuse A of the right-angle prism. The presence of a specimen on the prism surface modulates the phase front of the reflected light. Thanks to the prism presence, the object plane A optically appears to the camera, or to the plane H, at a certain angle of inclination, so an en face reconstruction result requires an algorithm that accounts for such an anamorphism.

Fig. 10

Apparatus for DH of total internal reflection. BS, beam-splitters; M, mirrors; L, lenses; A, object plane; H, hologram plane. Image is from Ref. 65.

In Fig. 11, the numerical correction procedure is depicted. The sample, Allium cepa (onion) cells, resides on the prism face and provides a direct image as shown in Fig. 11(a). With the addition of the reference beam, the CCD camera captures the hologram created by the superposition [Fig. 11(b)]. At that point, the hologram is processed into Fourier space, including filtering [Fig. 11(c)]. The complex array comprising the angular spectrum is then transformed back into real image space, yielding both the amplitude and the phase information [Figs. 11(d) and 11(e)]. If the untilting process is included in the reconstruction, the results are depicted in Figs. 11(g) and 11(h). In Fig. 11(f), a typical en face direct image of onion tissue is presented for comparison.

Fig. 11

Process of digital holographic microscopy with untilt via the angular spectrum method: engineering run with onion tissue (A. cepa). (a) Direct image with tilt. (b) Hologram. (c) Angular spectrum filtering first-order peak. (d) Amplitude image reconstruction with inherent tilt. (e) Phase image with inherent tilt. (f) Typical en face direct image of A. cepa. (g) Untilted (and transposed) amplitude. (h) Phase image. Image is from Ref. 66.

A 3-D version of this approach was introduced by Onural.⁶⁸ He used the impulse function over a surface as a tool, converting the original 2-D problem to a 3-D problem. Even though its formulation is analytically correct, it is proposed only for the continuous case.

Another method has been suggested by Lebrun et al.⁶⁹ to extract information about a 3-D particle field in arbitrary tilted planes by DH. In particular, he used wavelet transform to reconstruct small particles in a plane whose orientation is arbitrary as specified by the user. The pixels, whose 3-D coordinates belong to this plane, are selected and juxtaposed to rebuild the particle images.

More recently, a partial numerical Fresnel propagation technique of the complex wave has been proposed⁷⁰ for tilted image planes refocusing, and some solutions are used to reduce the influence of aliasing and Fresnel diffraction in the process of numerical reconstruction. A scaled Fourier transform is used instead in Ref. 71 to calculate light diffraction from a shifted and tilted plane. It seems to be faster than calculating the diffraction by a Fresnel transform at each point, see, for instance, Ref. 72, and this technique can be used to generate planar holograms from computer graphics data.

To simplify FTM and, at same time, solve the pixel size consistency problem, Wang et al.⁷³ presented a GPU-based parallel reconstruction method for EFIs of tilted objects. In summary, they used fast Fourier transform pruning with frequency shift combined with coordinate transformation. Their method has high imaging precision and speed, but it requires GPU assistance and some specific knowledge.

Generally, existing numerical methods for refocusing between inclined planes need a priori knowledge of the input scenes, such as the object size, and the average reconstruction tilting angle or distance, to properly adjust the EFI algorithm. Such a priori knowledge is easy to achieve in an academic experiment, but it is usually unknown for real experiments. In Ref. 74, Kostencka et al. proposed an appropriate tool for automatic localization of a tilted optimum focus plane. The method is based on the estimation of the focusing condition of the optical field by evaluating the sharpness in its amplitude distribution. The developed algorithm is fully automated. It consists of two major steps: first the rotation axis is localized from the map of local sharpness and then the angular orientation of the image plane is derived by maximizing the focus of optical fields reconstructed in many subsequent tilted planes.

In the case of a highly tilted plane or 3-D shapes with high gradients, the strategies described so far have encountered several problems. For DH in microscopic configuration, two reconstruction algorithms are presented by Kozacki et al.⁷⁵ The first is an extension of the well-known thin element approximation for tilted geometry, which can be applied to the case of large sample tilts, but it requires the sample numerical aperture to be low. The second one is called the tilted local ray approximation algorithm, and it is based on the analysis of local ray transition through a measured object. The authors proposed a modified algorithm for the numerical propagation between tilted planes, which can be applied for the shape characterization of tilted samples with a high shape gradient.

3.2.2.

Phase plate

In conventional microscopy, another possible solution for extended DOF is wavefront coding. This method was introduced by Dowski et al.^8–10 more than a decade ago. Wavefront coding introduces a known, strong optical aberration that dominates all other terms, like defocus. This circumstance causes the optical system to be essentially focus-invariant over a large range, so straightforward computational tools can be used to recover image information.

Under this imaging paradigm, several variants have been proposed.^12,76,77 Quirin et al.⁷⁷ have used a wavefront coded imaging system coupled to a spatial light modulation (SLM)-based illumination system, see Fig. 12, to image fluorescence from multiple sites in three dimensions, both in scattering and transparent media.

Fig. 12

Design of extended depth of field (EDOF) microscope. (a) Experimental configuration of the joint spatial light modulation and EDOF imaging microscope for 3-D targeting and monitoring. The detailed description of each component is described in Sec. 4.1 of Ref. 77. The phase aberration shown in (b) is the ideal diffractive optical element for the cubic-phase modulation and placed in an accessible region between L9 and L10 without affecting the illumination pupil. The experimental PSF of the imaging system is presented for the conventional microscope in (c) and the EDOF microscope in (d). The 3-D volumes in (c) and (d) represent the 50% intensity cutoff of each axial plane and the axis units are in micrometer.

For example, experimental results for the 3-D SLM illumination in transparent media, with both the conventional and extended DOF microscope, are shown in Fig. 13. In this case, a sample is translated axially $-500\mu \text{m}\le \delta z\le +500\mu \text{m}$ from the classical focal plane (defined as $dz=0$) in 4-μm intervals while another is held fixed in the focal plane (600 μm below the surface of the media), as shown in Fig. 13(a). The results from a conventional imaging microscope are presented in Fig. 13(b). For SLM microscopy, a rapid loss of imaging performance occurs as the illumination translates beyond the narrow focal plane. In contrast, the restored image from the extended DOF microscope is presented in Fig. 13(c), which shows a relative increase in the out-of-focus signal and tightly localized points, regardless of axial location. Although the results are clearly visible and noticeable, their extended DOF microscope requires a priori information on the target locations imprinted on the system by user.

Fig. 13

The 3-D illumination pattern is shown in (a). The results from imaging the 3-D pattern in bulk fluorescent material are given for the conventional microscope (b) and the EDOF microscope (c). Images are from Ref. 77.

In analogy to what is proposed by Dowski, Matrecano et al.⁷⁸ showed that a cubic phase plate (CPP) can be easily and conveniently included into the numerical reconstruction of digital holograms for enhancing the DOF of an optical imaging system and for recovering the EFI of a tilted object in a single reconstruction step. Moreover, they offered clear empirical proof through different appropriate experiments: the first one on an amplitude target and the others on biological samples. The advantage is in the possibility of avoiding the use of real optical components together with the related complex fabrication process required by a continuous cubic phase plate with a high phase deviation.

They propose to modify the numerical reconstruction algorithm. In particular, the hologram is multiplied by a numerical CPP, with a pupil function given by

Fig. 14

Phase distribution of a two-dimensional (a) and one-dimensional cubic phase plate (CPP), along the x-coordinate (c). In (b) and (d) are shown their mod-$2\pi$ representation.

In Eq. (25), a general 2-D phase delay is expressed as a function of both spatial coordinates. But, if an object is tilted by $\theta$ angle around the vertical $y$ axis during the reconstruction, the defocus varies along only the horizontal $x$ axis. Taking this into account, they modified the phase delay allowing it to become function only of the $x$ coordinate. Moreover, this consideration allows one to somehow interpret the cubic term influence within the reconstruction process. In general, quadratic terms^44,50 are used to compensate the defocus. In this case, it is not uniform but varies along the spatial coordinate. The effect is to change very little the areas near the focus distance $d$ very little, and, proportionally, to change the distant ones much more. The use of a numerical CPP, instead of a physical one, has the great advantage of increasing the system flexibility. In fact, by varying the amount of phase delay ($\alpha$ value) and the plate width ($R$ value), they can obtain an EFI notwithstanding the tilt angle or the image size.

In particular, considering a reconstruction algorithm and the introduced phase delay, through simple algebraic calculations, they obtain

The simulated phase distribution along $x$, with $1<x<2R$ and $\alpha =14\pi$, is shown in Fig. 14(c); its corresponding modulus $2\pi$ representation is shown in Fig. 14(d).

To show how the CPP introduction impacts the imaging in microcopy, they performed different experiments. Initially, they made an experiment with a Mach-Zehnder interferometer in Fourier configuration. The setup is a Mach-Zehnder interferometer in transmission configuration, and the laser wavelength is 0.532 μm while the CCD pixel size is 6.7 μm. A USAF resolution test chart was positioned in a tilted way with respect to the laser light illumination direction. The Fourier holographic configuration is such that the reference beam curvature matches that of the light scattered by the left side of the object (where the biggest number “1” is located). Consequently, this region is in-focus in the numerical reconstruction. On contrary, the target right side (where the number is “0”) is completely out of focus in the numerical reconstruction. In Figs. 15(a) to 15(c), the numerical reconstructions for the target tilted with an angle ${\theta }_y=50$, 55, and 75 deg are shown, respectively. The portion on the left side of the object is in focus, while the right part is out of focus and the focus gradually worsens going from the left to right. Instead, if a CPP is added before performing the numerical reconstruction (with an opportune choice of the parameter $\beta$), a DOF enhancement occurs, putting all tilted target details in good focus[see reconstructed images in Figs. 15(d) to 15(f)].

Fig. 15

Numerical reconstruction of the holograms for an object tilted with an angle of 50 (a), 55 (b), and 75 deg (c) as acquired by the CCD and after the CPP introduction, to obtain the EFI [(d), (e), and (f)]. Square insets are magnified view of circled areas for each color, respectively.

Moreover, they analyzed holograms relative to bovine spermatozoa, prepared by the Institute “Lazzaro Spallanzani” after fixation in seminal material suspension (see also Ref. 79). Figure 16 presents the quantitative phase maps, also in pseudo 3-D, between hologram reconstructions with, Figs. 16(c) and 16(d), and without the CPP, Figs. 16(a) and 16(b), for $d=200\mathrm{mm}$. Even in this case, since the focus is on the left, the contours of the spermatozoon on the right appear blurred and the tail is not well defined, see Figs. 16(a) and 16(b). Instead, in Figs. 16(c) and 16(d) (i.e., in the EFIs), both the head and the tail are distinguishable. Moreover, in pseudo 3-D phase reconstruction, a typical maximum into the head region, indicated by an arrow, is now clearly visible.

Fig. 16

Reconstructed quantitative-phase-map of bovine spermatozoa at $d=200\mathrm{mm}$, obtained without (first row) and with a CPP (second row). In pseudo 3-D reconstructions (d), the arrow indicates a phase maximum, which is visible only in reconstruction with a CPP.

As an application example in the case of arbitrarily tilted planes, the authors applied a numerical CCP to a 2-D grid, composed of poly-co-glycolic acid (PLGA) ink written onto a polydimethylsiloxane coated glass slide using a pyroelectrodynamic approach.⁸⁰ Figure 17(a) shows the grid amplitude reconstruction at a distance $d=400\mathrm{mm}$. At this distance, only the area along the diagonal, from left to right, is reconstructed in focus, while the PLGA fibers deposited along the lateral zones appear blurred and poorly defined, as highlighted by colored circles in Fig. 17(a). Through the correction of numerical CPP, the 2-D grid is reconstructed entirely in focus, see Fig. 17(b). The colored circles indicate that a depth of focus improvement occurred both along the horizontal and vertical directions. This shows the method can be applied to fix the anamorphism problem for arbitrarily tilted planes.

Fig. 17

(a) Grid amplitude reconstruction at $d=400\mathrm{mm}$. (b) Grid amplitude reconstruction at $d=400\mathrm{mm}$ after CPP correction. Colored circles show a depth of focus improvement along both directions.

The results definitely point out that this method allows one to obtain, in a simple and straightforward way, the EFI construction of a hologram recorded on a tilted plane.

3.2.3.

Geometrical hologram deformation

For tilted plane anamorphism correction, Paturzo et al.^81,82 proposed a significantly different approach. In particular, through a suitable quadratic deformation of digital holograms, they were able to construct the EFI of some targets, in general, to manage the depth of focus in 3-D imaging reconstruction.

They considered a spatial polynomial deformation.

If the sample is tilted only along one direction and the deformation is applied along the other one with suitable values of $\beta$ and $\gamma$ for the parameters, they recovered the EFI for tilted targets in a DH microscope. One example is displayed in Fig. 18. In this case, the tilted object is a silicon wafer with the letters “MEMS” written on it. The target is tilted with an angle of 45 deg with respect to the optical axis of the DH system. The deformation was applied only along the $x$ axis with $\beta =0.00005$ and $\gamma =0$. Figure 18(a) shows the reconstruction of the undistorted hologram at a distance of $d=265\mathrm{mm}$. It is important to note that the portion of the object with the letter “S” is in good focus, while the remainder is gradually out of focus. Figure 18(b) shows the reconstruction obtained on the quadratically deformed hologram, and it shows that now all the letters “MEMS” are in good focus, demonstrating that the EFI has been obtained. Figure 18(c) also shows the phase map difference calculated by subtracting the two holograms, indicating that the defocus tilt has been mainly removed.

Fig. 18

Quadratic deformation applied along the $x$ axis to a hologram of a tilted object. (a) First frame from the video showing how EFI is got by adaptive deformation. (b) EFI image. (c) Phase difference. Images are from Ref. 81.

The great advantage of this approach is in its extreme simplicity; it is direct and quite effective, but unfortunately the hologram transformation causes some deformations in the reconstruction. Anyway, digital holograms’ spatial and adaptive transformations are the key tools for creating a dynamic action of real-world objects, as proposed in Ref. 83. Though the EFI issue is not of major concern, this technique is interesting for handling the in-focus distance.

In their work, a 3-D scene is synthesized by combining multiple optically recorded digital holograms of different objects. The synthetic holograms can be given as input to any SLM array for optical reconstruction. The result is a 3-D scene truly observable of 3-D real-world objects projected in a volume in front of the SLM.

This technique allows full control in manipulating an object’s position and size in a 3-D volume with a very high depth of focus. 3-D dynamic scenes can be projected as an alternative to the complicated and heavy computations needed to generate realistic-looking computer-generated holograms.

3.2.4.

Selected applications

In this section, we want to compare some of the techniques described in the previous ones. The aim is to show how these methods work under the same conditions, proposing different applications.

Figure 19(a) presents the hologram of a target tilted with an angle ${\theta }_y=55\mathrm{deg}$. In Fig. 19(b), the amplitude reconstruction at $d=142\mathrm{mm}$ is shown. At this reconstruction distance, the image on the right side is in good focus, while the left one is out of focus. The results shown in Figs. 19(d), 19(e), and 19(f) are obtained by applying the method of angular spectrum rotation, numerical CPP, and quadratic hologram deformation, respectively. As one may notice, an extended focused image of the tilted target, in which the details are reconstructed in focus, is obtained. The differences between the first two techniques are almost nonexistent, while for the method using a spatial hologram transformation, the image on the left side still appears blurred and some deformation artifacts are visible. The reason for this poor performance is due to the high rotation angle and the numerical interpolations. For comparison, Fig. 19(c) shows the EFI obtained by a traditional sectioning and merging approach, as proposed by Ref. 23 for tilted objects. This result represents the ground truth for this type of application, because it is constructed by collecting only the in-focus parts in a single image. The results’ evaluation shows that the techniques using angular spectrum rotation and numerical CPP return results entirely comparable with the traditional ones, but with the unquestionable advantage that they are simple, direct, faster, and less error-prone.

Fig. 19

Comparison between exposed techniques. (a) The recorded digital hologram. (b) Digital reconstruction at $d=142\mathrm{mm}$. (c) EFI obtained by traditional sectioning and merging approach. (d) Reconstruction by angular spectrum rotation method. (e) Reconstruction after the numerical CPP introduction. (f) Reconstruction after a hologram quadratic deformation.

One can observe analogous results on the amplitude reconstruction of these biological samples. Figure 20(a) presents the hologram of two fibroblast cell lines NIH-3T3 flowing in a microchannel tilted $\sim 10\mathrm{deg}$. In Fig. 20(b), the amplitude reconstruction at a distance $d=280\mathrm{mm}$ is shown. Since the smaller cell moves away from the focus distance (again, on the left side of the image), it appears more blurred with respect to the bigger one. The amplitude reconstructions at same distance, but after the numerical corrections of angular spectrum rotation, numerical CPP, and quadratic deformation, are shown in Figs. 20(d), 20(e), and 20(f), respectively. After the use of the above methods, the edges of the second cell appear thinner and not more blurred along the entire microchannel length. The results obtained are almost completely superimposable with the EFI synthesized by a conventional approach in Fig. 20(c).

Fig. 20

Comparison between exposed techniques. (a) The recorded digital hologram. (b) Digital reconstruction at $d=-280\mathrm{mm}$. (c) EFI obtained by traditional sectioning and merging approach. (d) Reconstruction by angular spectrum rotation method. (e) Reconstruction after the numerical CPP introduction. (f) Reconstruction after a hologram quadratic deformation.