The measurement of flows in microfluidic systems or in blood vessels can be obtained either by means of single particle tracking1–3 or by applying cross-correlation methods on images. Both techniques rely on high-resolution and high-contrast imaging of the sample. Nonlinear excitation methods, in which tightly focused near-infrared pulsed lasers induce the fluorescence emission, have been recently introduced to reduce light scattering from the living tissues4 and increase in general the signal/noise ratio. However, two major phenomena still limit the signal/noise ratio in nonlinear microscopy: the autofluorescence of the tissues and the aberrations of the excitation wave front induced by its propagation through the highly inhomogeneous tissue.
Autofluorescence and scattering that increase the background in different amounts depending on the tissue conservation state4,5 can be reduced by increasing the excitation wavelength. The propagation of the light wave through the tissue increases the pulse width and warps the wave front shape. Both phenomena lead to a decrease of the two-photon excitation efficiency. The pulse width can be reduced by means of a prechirping unit. The wave front distortions can be partially corrected by adding a phase mask on the excitation wave front that precompensates for the tissue induced dephasing within the general approach of adaptive optics (AO).6
Our aim here is to exploit AOs methods for the correction of the optical aberrations in order to improve cross-correlation analysis of flows in biological samples. Cross-correlation methods, in which we measure the time-of-flight of fluorescent microbeads between observation volumes lying along the flow lines, require parallel (i.e., simultaneous) collection of the signals from sparse observation volumes with high pointing stability and high signal/background ratio. Our rationale is to use a single diffractive programmable element to obtain multispot excitation of the sample and to correct optical aberrations on the spots in order to improve the quality of the interspots cross-correlation function (CCF). Our results depend on the unique coupling of cross-correlation image analysis and aberration correction, achieved on a single optical path without any movable mechanical part. Therefore, we briefly review these two major research areas in the following.
Spatial light modulators (SLMs) have been widely used to change the image modality7 of optical microscopes as well as to correct aberrations. For example, Gharbi et al.7 implemented a simple method based on multiple carrier frequencies to reduce the chromatic aberrations in an incoherent illumination microscope. Hasler et al.8 developed a two-image recording method with digital postprocessing based on an SLM, which is equivalent to a phase contrast microscope. Lingel et al.9 developed an optical setup for phase retrieval, in which the object can be masked first in the intermediate image plane and then filtered in the Fourier plane by making use of a single SLM and no movable mechanical parts. Even more recently, Mastusmoto et al.10 described a simple algorithm to correct the large aberrations induced by the shape of the surface of the biological tissues in two-photon excitation microscopy with a dry objective.
The possibility to implement AOs corrections in raster scanning setups for nonlinear microscopy has been first reported by Neil et al.11 These authors presented an intensity-based sensor-less AOs method for two-photon raster scanning microscopy obtained by implementing a differential mapping of the phase of the object. Débarre et al.12 substantially improved on this scheme and demonstrated the feasibility of SLM-based aberration correction of two-photon images of biological specimens.
The deformable mirror and multiactuator deformable lens are possible alternatives to the SLM. Two general approaches to the wave front correction are reported in the literature: zonal correction and modal correction13,14. In the first case, the wave front is described as the superposition of basis functions, typically the Zernike polynomials, and the correction is achieved by changing their amplitudes. In the second case, the correction is performed on the back pupil plane of the objective, which is divided in S regions of interest. Deformable mirrors are not much sensitive to the light polarization and wavelength15 and allow for modal or zonal correction of the aberrations.13,14 Multiactuator deformable lenses easily allow for modal correction and depend on the light wavelength.16 The SLM offers a huge number of degrees of freedom for the correction of the aberrations at the cost of a slow refresh rate. It allows to apply modal or zonal corrections of the aberrations and to create customized illumination patterns, such as grids of spots, on which cross-correlation analysis can be easily performed.
Cross-correlation analysis of flows in microfluidic devices constitutes a wide research field with implementations that span from microPIV data analysis17 to flow mapping on single images.18 It has recently acquired great impact in cellular biology, where it is applied on 2-D images19,20 in order to characterize the flow field in the image plane.21 The cross-correlation analysis of the sample dynamics can be based on a multispots simultaneous excitation of the sample22–25 or on a spatially distributed and time multiplexed excitation imaging as in raster image correlation spectroscopy.8,26 Multifocal excitation and collection, occurring in parallel on a pixelated device, offer best results for the analysis of fast flows. Multifocal excitation was also used to increase the image acquisition rate or to gain a parallel acquisition on a sparse array of observation volumes. The laser beam was mechanically moved on the sample and coupled to an acousto-optic deflector27–29 or an SLM30 in order to obtain a multispot excitation of the sample. In these works, a fine adjustment of the position of the spots on the spines was done by means of galvanometric mirrors. However, the pointing stability was not assessed directly or by means of a cross-correlation analyses.
Based on these works, we thought that only scan-less setups with no movable parts could offer the parallel acquisition needed to perform cross-correlation analysis. Some of us preliminary tested this assumption in collaboration with D’Angelo et al.31 on arrays of neurons in acute cerebellar slices of rats. The scan-less parallel acquisition developed in Ref. 31, based on the use of an SLM, offered the possibility to draw connectivity maps of neurological activity in an ensemble of communicating neurons32 in the granular layer of acute rat cerebellar slices. The images were collected by shifting an array of laser spots on the sample, by collecting the signal of the whole array of spots through a sensitive EM-CCD camera, and by superimposing these signals on a single map. The number of spots in the array and the shifting step size determined the frame rate and the image resolution: the larger was the number of spots per array, the lower was the image acquisition time. However, the number of spots per array cannot be raised without rapidly degrading two-photon excitation efficiency, a fact that could be partially overcome by optimizing the laser beam shape. In Ref. 31, the SLM was not exploited to correct aberrations of the optics that could result in an increase of the two-photon excitation efficiency. In addition, no attempt was done to measure the CCFs between the spots of the array.
The novelty of the present report with respect to Ref. 31 is, first, to substantially improve the localization of the spots on the sample (or camera) plane and implement a more accurate biquadratic mapping method between the SLM and the camera planes. This is essential to obtain accurate mapping of flows and in general to reduce artifacts in cross-correlation analysis on sparse samples. Second, we implement on the same SLM an optical aberration correction algorithm that increases the performances of the microspectrometer for combined morphological and fluidodynamics studies. The SLM is used to shine rectangular arrays of spots, to correct their optical aberrations, and to shift them on the sample so to collect two-photon excitation images of biological specimens. In a second step, the SLM is used to shine arbitrary patterns of aberration corrected laser spots, on which we compute the CCFs that provide the measurement of the flow velocity. This setup is, therefore, capable to collect aberrations-corrected two-photon images, to point at specific regions of interest with high accuracy, and to increase the sensitivity of the cross-correlation analysis of flow velocities. We combine for the first time the correction of wave front distortions to an excellent pointing accuracy on arbitrary geometries of excitation volumes, achieving a fivefold increase in the signal-to-noise ratio of the acquired CCFs, a twofold decrease of the uncertainty and a threefold increase of the accuracy in the evaluation of the flow speed.
Materials and Methods
The optical setup consists of a custom-made upright two-photon holographic microscope (Fig. 1). Two-photon excitation is obtained in the samples by a Ti:Saph laser (Tsunami, Spectra Physics, California) providing 1 W maximum power at . The polarization of the laser light is perpendicular to the optical table. The laser is collimated and expanded by a beam expander ( and ) and impinges at 9 deg on the reflective SLM (X10468-7, Hamamatsu, Japan), which allows to encode phases in the range 0 to on the beam wave front with resolution ( size) and 8-bit pixel depth. The SLM plane is conjugated through a beam reducer ( and ) to the entrance pupil of a microscope objective ( XLUMPLFLN, Olympus, Japan). An arbitrary intensity distribution in the focal plane is obtained by encoding a phase mask on the SLM.33,34 The contribution of the zero-order diffraction from the SLM onto the sample plane is minimized by superimposing a Fresnel lens (FL) pattern on the calculated phase masks,35 while slightly displacing the position of the second lens of the beam reducer from the telescope condition.31
The sample fluorescence is discriminated from the excitation light through a dichroic mirror (DM) (HC670SP, AnalysenTechnik, D) and a bandpass filter (ET680SP-2P8, AnalysenTechnik, D). An image of the objective focal plane is formed by a LT () on a Hamamatsu CMOS Orca Flash 4.0 Camera (pixel size ).
SLM-Based Illumination Pattern Creation
Aberrations that detrimentally affect the resolution and the contrast of microscope images can be corrected by means of AO with a variety of active diffractive or reflective elements.11,36,37 Here we exploit an SLM to create custom grids to implement the cross-correlation methods on a multiple spots basis and to acquire images with no movable parts in the optical path similar to Ref. 31. Differently from that work, we combine here these features with the possibility offered by the SLM to correct the wave front distortions on different regions of interest throughout the field of view.
A grid of spots is created in the sample plane by loading onto the SLM a phase pattern computed through the weighted Gerchberg-Saxton (GSW) algorithm.33,34 The resulting phase pattern (GSW-phase) is modulated between 0 to before applying it onto the SLM. The grid intensity profile projected onto the sample plane is rescaled on the two axes, since a discrete 2-D Fourier transform is applied on a rectangular domain (the SLM plane) with different sizes (). We calculate two conversion factors and , respectively, equal to the ratios between the mean horizontal and vertical spacing between two spots in the SLM domain and in the camera plane. These factors allow calculating the horizontal and vertical spacing in the SLM domain to produce a squared grid in the sample plane [see Eq. (10)].
One of the major issues related to the use of the SLM is the so-called zero-order background.35 A percentage () of the collimated light incident onto the SLM is not modulated by the pixelated structure and produces non-negligible levels of diffused light in the sample plane, which results in a limited signal-to-background ratio of the SLM-generated illumination pattern. In order to reduce the zero-order contribution, we placed the L3 and L4 lenses [Fig. 1(a)] at a distance shorter than the sum of their focal lengths of a quantity , , and we add a quadratic phase pattern (FL) to the GSW-phase.31 The FL has the following expression:Fig. 1(a) ( = refraction index = 1). An FL phase-modulated wavefront passing through the lens L3 is focused on a plane placed at a distance with respect to the lens focal point, but corresponding to the focal point of the lens L4. Therefore, it will leave the beam reducer as a collimated beam. The zero-order contribution will instead diverge since it is focused after the focal point of the lens L4 along the optical path.
The image reconstruction algorithm is based on a grid of spots that is generated by the SLM and shifted over the field of view. The cross-correlation method is applied between couples of spots on the sample plane. In both cases, it is fundamental to identify with high accuracy of the position of the spots in the camera plane. Compared to Ref. 31, we improved substantially the accuracy of the localization of the laser spots on the sample by means of blob-detection algorithms (BDAs)38 and of a biquadratic mapping between the SLM and the camera planes. We checked that after a careful calibration of the sample-to-CMOS planes mapping the spots positions on the camera could be identified for a wide variety of grid parameters.
We start the calibration algorithm by projecting a grid with known features onto a Rhodamine plastic slab and acquiring the fluorescence signal with the CMOS camera. In order to identify the groups of pixels that correspond to each spot of the grid, we exploited a BDA:38 a blob is a group of connected pixels that share some common properties, e.g., grayscale value. The BDA works as a spatially resolved bandpass filter on the signal levels between a minimum and a maximum threshold: in our case, the maximum threshold was set to 255 digital levels (8-bits depth). It must be noticed that the position of the blob in the BDA is a real variable not limited to the integer values that identify the real pixels. Since the grid was typically affected by an intensity gradient, we first subtract from the grid image the zero-order contribution obtained by deleting the FL mask from the GSW algorithm. The image was then split in several subregions, the BDA-detection algorithm was applied separately on each of them, and the detected spots were then stored in an array. Finally, for each position stored in the array, we copied a regions of interest (ROI) centered at the putative spots from the image of the zero-order subtracted grid image. We then applied a Gaussian filter and autocontrast to the whole new image before iteratively applying the BDA for increasing values of the minimum threshold from 0 to 255 until we found a number of blobs equal to the number of spots of the grid. As a final step, we took as putative spot position in the camera plane the maximum pixel coordinates value inside the ROI centered on the blob position.
Prediction of spots position
In order to estimate the position of the spots of a grid with features different from the calibration ones, we adopted a 2-D second-order parametric model39 that describes the mapping of the SLM plane onto the sample plane. This model allows us to retrieve the spots position without applying the BDA each time we change the grid shape. The transformation between the position of a single spot in the SLM domain and the camera domain can be expressed as
We adopt a matrix notation, where is a matrix ( is the number of spots in the grid) whose rows contains the coordinates of the ’th spot in the camera domain, is a matrix whose ’th row contains the products , and is a matrix that contains, in each row, the expansion coefficients of and into polynomials [Eq. (2)]. Once the matrices and are obtained from the mapping calibration procedure, the transformation matrix is simply calculated as39
The matrix is computed once from the data obtained in the mapping calibration procedure for a specific choice of the grid parameters. For different grid spacings in the sample, characterized by a matrix, the position of the new spots in the camera domain could be estimated as
Our aberrations-correction algorithm is based on a sensorless AO scheme.11,17,40,41 We assume that a wavefront can be expressed as a weighted sequence of basis functions or aberration modes and that the image quality can be restored by maximizing a metric that depends on the expansion coefficients of the wave front in terms of the modes.
Aberrations representational model
The wave front propagating toward the sample plane can be described at the entrance pupil as a weighted sum of basis functions:42 We needed here to extend this basis to the Cartesian representation of the Zernike modes43 due to the rectangular symmetry of the SLM. In order to do so we followed the algorithm described in Ref. 43 and adopted a Noll’s index scheme.44 We have limited the aberration corrections to the 11’th Zernike polynomial (primary spherical).
The versatility of our system in creating custom illumination pattern is challenged by its low-speed performance in acquiring images. For this reason, we decided to build a wave front metric based on the intensity measured at selected spots on the sample plane, instead than on the whole image as done in other studies11,45,46 based on raster scanning or wide field image acquisition. As we discuss in Sec. 3, this algorithm could also be modified to account for the correction of nonisoplanatic aberrations.47,48 Our aim is to obtain bright and small spots on a low zero-order background after the application of the aberrations-correction algorithm.
Our metric is based on the comparison of the intensities integrated over ROIs (size ) centered on each of the spots of the grid () with the background measured by calculating the mean integrated intensity over four ROIs (size ) set at the vertexes of a square centered on the grid central spot whose side is equal to the spot–spot distance. Throughout this report , unless otherwise specified. By limiting the computation of the background to the central spot, we are overestimating the effect of the zero-order reflection from the SLM that is the primary source of background and fades at the edges of the field of view. Therefore, we computed the metric over the whole field of view covered by the spots as
A common assumption, well posed in many practical situation, is to describe the wavefront as a finite sum of Zernike modes with amplitudes and to approximate the metric with a paraboloid surface in the vicinity of its maximum:11,40,45,49
In Eq. (7), and are the constants. This function has a clearly defined maximum, but each term of the sum depends upon the amplitude coefficients of two aberration modes. Since the matrix is not strictly diagonal, the optimization of the metric is not straightforward. It is possible to diagonalize the matrix17,49 and get, at high computational cost, a correction improvement that depends on the amount of intermode coupling. As proposed in previous works,11,17,45 we adopted two searching algorithms based on the maximum parabolic interpolation.50 The first one searches sequentially the maximum of the metric as a function of the individual ’th mode amplitude, while depressing the effect of the other ( if ). This leads to the best mode amplitude (BMA) for each of the considered Zernike polynomial. We then sum the corrections of all the modes at their BMA. In an alternative algorithm, we searched the global maximum of the metric by means of the gradient descent method (GDM). In this algorithm, we initialize all amplitude modes to zero, numerically calculate the gradient of the metric by means of the Newton’s difference quotient method, exploit the backtracking line search algorithm51 to define the optimum step to update all the amplitude modes, and iterate this procedure until the absolute value of the gradient is lower than an arbitrary chosen cut-off.
Experimentally, the two methods produce similar results in the samples investigated here. This means that the mode that at its BMA produces the highest correction to the metric is typically the mode whose relative derivative at the starting point of GDM is larger. In general, by independently maximizing the metric on each mode, we obtain an estimate of its contribution to the wave front aberrations. Instead, the GDM provides an aberration correction pattern that involves the contribution of all the modes and may lead to a slightly larger improvement of the metric.
Our aim is here only to characterize the morphology of the sample before the flow measurement without any moving optical parts in order to be able to position with high accuracy of the observation volumes for the cross-correlation measurements and to measure the flow velocity. The image is acquired through a shift-and-snap procedure.31 We project a spots grid onto the sample plane. Then the signal of each spot is read and stored in the corresponding pixel of the image [the position of each spot in the camera plane can be predicted by Eq. (4)]. We shift the grid in the sample plane along horizontal and vertical directions and iterate the procedure, building up the final reconstructed image. The grid is shifted along the horizontal direction by adding a prism phase function to the grid pattern profile:Fig. 1(b)]. For the shift along the row direction , while for the shift along the column direction .
The number of shift movements along horizontal and vertical directions (respectively, and ) is determined by the number of spots and the size in pixels of the reconstructed image:Fig. 1(b)]. The spacing between spots in the SLM domain , depends on the number of shifts and the pixel size in the image plane ( and ): 2.3) and , are the linear sizes of the camera pixel in (in our case ). The shift of the grid (in terms of SLM pixels in the SLM plane) can be determined as
Finally, the signal read from the spot in the ’th column (from left to right, ) and ’th row (from top to bottom, ) of a grid that has been shifted times along the horizontal direction () and times along the vertical direction () is assigned in the synthetic reconstructed image to the pixel with indices :
When we reconstruct an image, we usually integrate the signal from a ROI centered in every putative spot position, in order to minimize the fluctuation of the spot intensity. The description of the grid shift algorithm given above in the SLM plane by means of the spot–spot distance and the shift parameters can be converted in the sample plane by means of equivalent parameters called and used in Sec. 3.
The flow velocity can be measured from the analysis of multiple CCFs. The two-volumes normalized CCF is defined as4,6,52
In Eq. (14), is the diffusion time, is the diffusion coefficient of the colloids, is the beam waist, , accounts for the volume elongation along the optical axis, and is the time of flight between the observation volumes. When the flow axis is collinear () to the interspot distance , Eq. (14) can be approximated by a simple Gaussian function in the lag time with maximum at the lag time , for small values of the times ratio . Numerical simulations proved6 that for the discrepancy between and is lower than 0.5%. If the interspot vector is not collinear to the flow velocity, the argument of the exponential function in Eq. (14) does vanish exactly, the lagtime of the maximum of the CCF is displaced with respect to as . Moreover, the amplitude of the CCF decreases as a function of the noncollinearity angle as , where is the angular size of the downstream volume as seen by the upstream volume. The correction to the position and the amplitude of the maximum of the CCF is second order in the misalignment angle .
Equation (14) provides only the flow speed, through the measure of , in which the maximum of the CCF falls. In order to map the flow velocity, we projected grids of various symmetries on the sample plane and calculated the CCF between each pair of neighboring spots. For each spot (called hereafter C-spot), we retrieved the neighboring spot (called hereafter NN-spot), for which the CCF shows the highest cross-correlation amplitude and assign to the C-spot a velocity whose amplitude is the speed calculated from and whose direction lies along the vector pointing from the C-spot to the NN-spot.
In an effort to enhance the signal/noise of the cross-CCFs, we applied clipped correlation algorithm,53 in which the time trace of the signal collected on an ROI is digitalized according to a threshold value, Th. All the time points for which the signal exceeds Th are set to 1, otherwise they are set to 0.
Spots Identification and Spot Prediction Algorithm
Both image reconstruction and cross-correlation analysis of flows require an accurate identification of the spots in the grid. We validated the stability of our spot identification algorithm on a variety of grids with different parameters (spots number and spot–spot spacing) and for different values of the signal-to-noise ratio. An example of the algorithm output is given in Fig. 2(a), for a grid (spot–spot spacing in the sample plane). In this example, the signal-to-background ratio was and the pixel with the highest intensity coincides within one pixel with the putative spot position obtained by means of the BDA algorithm (see Sec. 2.5.1). By analyzing a series of low signal/noise ratio images obtained by reducing the laser power or the camera exposure time, we checked that the BDA algorithm retrieves the spots position well within one pixel (average positioning ) for signal/background ratios as low as 1.35 [see Fig. 2(b)]. The accuracy of the spot prediction algorithm was validated also on shifted grids by applying the spot retrieval algorithm on grids projected onto a Rhodamine plastic slab, finding similar accuracy over the whole range of shifting values employed here.
Algorithm for correction on a single array of spots
We analyzed and corrected the optical aberrations affecting a grid of spots projected onto a Rhodamine plastic slab by choosing at first to maximize the metric [Eq. (6)] independently on each Zernike mode and tested modes up to (primary spherical). The first estimate of the aberration corrections is the ratio between the metric of the uncorrected and corrected image. In order to enhance the image quality, we superimposed the corrections for the modes that exhibit the metric improvement . The improvement in the two-photon image of a grid (interspots spacing ) projected onto a Rhodamine slab can be judged visually from Fig. 2(c). The analysis of the images [line profiles reported in Fig. 2(d)] indicates that the relative improvement of the signal-to-background ratio between the raw () and the corrected spots () lying on the dashed line in Fig. 2(c) is .
We studied the dependence of the aberrations on the position of the grid within the FOV and the possibility to apply different corrections over a wide FOV. This was done by shifting a grid in different regions of the FOV onto the Rhodamine slab [Fig. 3(a)]. The correction amplitude of each mode [ in Eq. (5), ] changes over the ROIs as it is reported in Fig. 3(b). The analysis reported in Fig. 3(a) and 3(b) could be performed for an extended field of view in a thick biological sample, in which scattering from different regions of the out-of-focus portions of the tissue produces nonuniform aberrations, and therefore, requires spatially dependent corrections.
Signal/background and spots size
The signal/background of the grid (average of the maximum of the spot divided by the background between two adjacent spots) is increased typically by a factor of times by the application of the Zernike modes corrections. However, the efficiency of the aberrations correction is largely affected by the spot–spot spacing in the grid. The signal/background ratio decreases rapidly with the reduction of the spacing among the spots of the grid: by changing the spot–spot spacing from 67 to , the signal/noise ratio on the corrected image decreases from to [Figs. 3(c)–3(e)].
The effect of the aberration correction on the single spot is a considerable reduction of its full width passing from to for the same number of spots in the matrix [, Fig. 4(a) middle plot and Fig. 4(b)]. However, the aberrations-correction effect on the width of the spots depends on the number of spots of the grid at fixed spot–spot distance [Fig. 4(a)]. We report as an example in Fig. 4(a) the trend of the spot width as a function of the number of spots for images taken at spot–spot . By changing the grid from a to a matrix, the full-width at half-maximum decreases from to [Fig. 4(a)].
Image Reconstruction and Correction
We tested our image reconstruction and aberration correction algorithm on a sample of Convallaria (Convallaria majalis stained with fast green safranin, AS3211, Leica, D). By projecting a grid of spots and shifting it over the sample in the and directions, we were able to reconstruct the nonlinear excitation autofluorescence image (excitation at ) of the sample. Figure 4(c) shows an example of the images obtained before the aberrations correction, by means of a spots grid (spacing in the sample plane) shifted with steps in the sample plane. The grid shift corresponds to CCD pixels in the camera plane. The grid rectangular distortion was quantified and corrected with correction factors and in the and directions, respectively. For each pixel of the reconstructed image, the intensity was computed as the mean of the signal on a mask centered in the corresponding spot of the grid employed in the specific image reconstruction step. The efficacy of the spot array aberration correction can be visually estimated over the whole field of view from the comparison of Figs. 4(c) to 4(d) and better evaluated from the zoom reported in Fig. 4(e), where uncorrected and aberration-corrected images are juxtaposed on the same image. To correct the optical aberrations affecting our setup, we evaluated the metrics of Zernike modes from 5th to 11th on a Rhodamine slab, optimizing all the modes simultaneously (gradient search GDM, see Sec. 2.6.2) on a fixed grid of spots. We then collected the image by summing the aberrations-correction phase pattern to the grid phase pattern at each acquisition step. The mean signal-to-noise ratio increases more than twofold (), as can be judged from the profiles reported in Fig. 4(f). The resolution of the structures of the sample is clearly better after the aberrations correction as can be judged from the possibility to resolve the y-shaped structure in Fig. 4(e) along the solid and dashed white lines for the aberration corrected (AC) and uncorrected (BC) images at about and , respectively, [Fig. 4(f)]. The full-width at half-maximum of the two peaks that are clearly discernible in the aberration corrected image profile is . High-resolution imaging of the same sample features with a Leica SP5 confocal microscope (resolution ) gives a width for these structures of . These values indicate an optical resolution of the present setup of .
Cross-Correlation on Grid of Spots for Flow Mapping
After getting an image of the sample, the grid array can be positioned at a specific position with an accuracy that is better than a single pixel [see Fig. 2(b)] and any bright object moving through the spots can be detected with a resolution of 10 ms, determined by the camera acquisition time. We tested the accuracy with which we can measure the flow speed on microchannels of varying geometries, in which we injected microbeads suspensions at fixed speed values. The microbeads passing through the spots are detected as bursts of signal on the camera image. The density of the microbeads is low enough to provide distinct sparse bursts of signal on the time profile of the time stack of images.
Two geometries of microchannels were studied: a straight square cross-sectional glass microchannel ( size) and a horseshoe-shaped microchannel ( size) obtained by PDMS molding. We exploited the full capability of the SLM in creating custom illumination profiles and measured the cross correlation of the signals collected from the two volumes (the signal was integrated on a area). The principle of cross-correlation spectroscopy52,54 requires the synchronous collection of the signals from two laser spots along the flow, a possibility offered by the SLM that also allows to maximize the signal/noise ratio by correcting the optical aberrations on the selected spots.
At first, we applied the cross-correlation method on couples of spots lying along the flow of a straight glass microchannel. The two-photon excitation depends on the square of the excitation intensity. By multiplexing the laser power on a grid, we reduce the excitation intensity on each spot of the grid by a factor scaling as . So, as a first choice, a rectangular grid of spots ( and , sample plane) was projected on the sample. We estimated the direction of the flow by cross correlating the central spot [red dashed circle C’, Fig. 5(a)] with each of its nearest neighbors [orange circles, from A to E, Fig. 5(a)].
The results shown in Fig. 5(b), respectively, obtained before (BC) and after (AC) the aberration correction, indicate that in both cases one of the CCFs (corresponding to the “C” spot) has dominant amplitude. The amplitude of the cross correlation was approximately threefold higher for the AC case () than for the BC case (). We also found a systematically sharper correlation peak ( reduction in width). We assign, therefore, the flow direction to the vector joining the corresponding couple of spots [from C’ to C in the example given in Fig. 5(a)].
The amplitude and the shape of the cross-correlation peak depend also on the threshold parameter, Th, and the size of the ROI over which the spot signal is averaged. The use of threshold values increases the amplitude of the cross-correlation peak of almost threefold [Fig. 5(c)]. As to the ROI size around each spot, the minimal choice gives the largest amplitude of the CCF [Fig. 5(d)]. However, the result is also very sensitive to the exact positioning of the ROI on the irradiation spot. On the other hand, by increasing the ROI size above , the amplitude of the CCF decreases markedly [Fig. 5(d)]. Therefore, we set the threshold value at and adopted a ROI in the following analyses. This choice will depend on the brightness of the dye labeling the tracers and the background of the flowing fluid.
We can monitor the flow speed in a wide range of values by measuring the time of flight from the maximum of the CCFs [Fig. 5(e)] between two spots. We tested it for three different values of the velocity (120, 400, and ) retrieving the values set by the syringe actuator with an accuracy (the difference from the expected value) better than 3% and an uncertainty (variability of the measurement) of at most 6% (, , and ). The relative increase in the signal/noise obtained by correcting the aberrations is more and more effective the faster is the flow speed, as can be seen in Fig. 5(e) by comparing the dashed (corrected) to the solid (uncorrected) lines.
In order to refine the flow field mapping, we can device more elaborate illumination patterns with a larger number of spots. In this case, the correction of the aberrations was even more essential due to the rapid decrease of the two-photon excitation efficiency per spot with their number. An example on a array is given in Fig. 5(f) ( spots array, spacing , and in the sample plane). The analysis of the cross-correlation amplitudes (data not shown) indicates that the preferential flow direction is from the central spot to the “D” spot, within an accuracy angle determined by our choice of the array geometry that in the case of Fig. 5(f) . By means of a wider array, we can also reduce the uncertainty on the flow speed that can be measured for increasing distances between the spots [as sketched in Fig. 5(g)]. An example given in Fig. 5(h) suggests that by interpolating the time-of-flight time as a function of the spots distance the uncertainty on the speed can be reduced to at least 0.4% ().
The possibility offered by the SLM to create arbitrary patterns on the sample can be further exploited to characterize flow patterns. To illustrate this feature, we have employed a tightly curved microchannel [channel width ; channel inner radius ; Fig. 6(a)] obtained by PDMS casting, sealed on a glass slide. The presence of uncured PDMS on the glass slide that was used to seal the microchannel, induced additional aberrations on the light wave front. We have computed the CCFs between couples of spots in a array [Fig. 6(b), , , field of view (a) in Fig. 6(a)]. We found non-negligible cross-correlation amplitudes for all the couples of spots lying along the horizontal lines in Fig. 6(b). In this case, the correction of the aberrations is more effective than for the glass capillary: we could increase by a factor the signal/background in the time-of-flight peak of the function by applying the aberrations-correction algorithm [Fig. 6(c)]. The flow speed uncertainty decreases almost threefold from 1.8% [, open squares, Fig. 6(c)] to 0.7% [, filled squares, Fig. 6(c)].
These results are confirmed also in different positions of the microchannel. For example by averaging over all the pairs of spots lying along the lines 1 to 5 in the field of view (b) [see Fig. 6(a)] that lies closer to the channel wall (at about from the channel axis), we find [Fig. 6(d), solid lines], that agrees well with the estimate .
We have exploited the SLM also to shed a circular array of spots [shown as maximum projection image in Fig. 6(e)] on the sample plane at the position marked by the field of view (c) in Fig. 6(a). Such patterns could help in discriminating the direction of the flow for curved flow lines. The results of the cross-correlation measurements between the central spot and each of the nearest neighbor spots [Fig. 6(f)] indicate that the flow points toward the spot “C,” in agreement with the shape of the microchannel, with a best fit value of the speed , not dissimilar from the speed in the field of view (b).
Our algorithm is based on grids of spots shined and shifted regularly on the sample only by means of the SLM. The accuracy in the prediction of the spots position of a grid projected onto the sample plane is fundamental to achieve high-quality image reconstruction and accurate positioning of the observation volumes for the computation of the CCFs. We have shown that our algorithm, based on a combination of the BDA and image filtering, retrieves the spots position within (well within the pixel size) for signal/background ratios as low as 1.35 [see Fig. 2(b)].
Our aberrations-correction algorithm is based on an intensity metric averaged over the spots of the grid [Eq. (6)] that spans the whole field of view. We can routinely reach a relative improvement of the signal-to-background ratio between the raw and the corrected spots between 3 [Fig. 2(c)] and 5 [Fig. 3(d)] over the whole field of view. However, the spot–spot spacing in the grid affects the improvement of the spot signal/background obtained by means of the aberrations correction. It decreases from to when the spot–spot distance decreases from 67 to . Obviously, we would like to keep the spot–spot distance as small as possible, and the number of spots in the array as large as possible in order to reduce the acquisition time. The improvement in the excitation efficiency obtained through the aberrations correction allows us to use arrays as large as (see Fig. 4).
The corrections of the aberrations can be influenced also by (anti)correlation effects in the modes or in the image regions. Regarding the first possibility, we have tested our algorithm for the aberrations correction on a grid of spots for a possible cross-talk between pair of modes and . This parameter could be measured through the ratio of the out-of-trace terms, of the matrix and a geometric average of ’th and the ’th modes term . However, we estimated experimentally the degree of coupling of the ’th mode with all the others, from the ratio of the metric obtained by correcting for the ’th mode only over the metric value obtained by correcting all the modes (GSD algorithm). In our experiments, this ratio was typically , indicating that the coupling between the modes was limited.
Regarding the second possibility, we should consider that the optical aberrations may differ substantially over the field of view. The contribution to the metric value of brighter spots is higher with respect to the dimmer ones that typically lie at the margin of the FOV: difference in the spots brightness can be serious for large () FOV. Moreover, we expect that living tissues display anisoplanatic aberrations.47,48,55 We have tested the optical setup for possible anisoplanatism by performing corrections on arrays on adjacent ROIs covering a wide field of view (see Fig. 3). Operatively, on each ROI, we could compute the amplitudes of the Zernike polynomials that could be used to correct the optical aberrations locally and then reconstruct the wide field of view by stitching together the ROIs. However, a few general conclusions can also be drawn. The overall correction for each ROI shown in Fig. 3(a), measured as the average of all the Zernike amplitudes over the ROI, , does not increase dramatically when moving from the center of the FOV () to the periphery . Moreover, the most effective modes as measured by the mode amplitude averaged over the 9 ROIs, , are , 5, and 10 (, 1.0, 1.2, respectively). This type of analysis can be in principle used to evaluate operatively the anisoplanatism over the desired field of view.
The interplay of the limited spatial frequency resolution and content of the SLM with the zero-order reflection produces unexpected effects in the aberration correction as we can judge from the analysis of the spot size and the signal/background ratio of the laser spots array reported in Figs. 3 and 4. In fact, we observed an apparent reduction of the spot size if measured on top of the background level [the full-width at half-maximum of the spots measured on top of the background decreases from to , Figs. 3(c)–3(e)] when we decrease the spot–spot distance in the grid from 67 to . This behavior, which is opposite to that observed for the signal/background level [see Fig. 3(c) inset], is an artifact related to the increase of the background level on the grids at low spot–spot spacing. This background increase is probably due to the discrete nature and the finite size56 of the hologram encoded on the SLM that prevents to accurately encode the hologram containing the Fresnel zone mask needed to correct for the zero-order reflection and a set of closely spaced spots.
Image Reconstruction and Aberrations Correction
Having tested the accuracy of the spot position retrieval and the aberration correction algorithm, we moved to combine them in the image reconstruction algorithm. The results shown in Fig. 4 indicate that modal corrections of the optical aberrations can be implemented in the SLM-based imaging algorithm based on the “shift and snap” method.31 The total acquisition time for the image shown in Fig. 4(e) (, 400 images) is about 25 s (10-ms exposure time per frame).
The major limitation to the image acquisition rate comes from the refresh time of the SLM, for the Hamamatsu X10468-7 used here. The sample brightness would not limit the image acquisition rate substantially, unless for very dim samples. In fact, from a in a sample with a protein concentration and a count per molecule (for a 10% collection efficiency), we should get or 240 electrons per pixel of an image acquired at 1 kHz (conversion factor for Hamamatsu Orcad camera), corresponding to an acceptable level of noise . The minimum theoretical image acquisition time is then largely limited by the SLM refresh time and the synchronization between the SLM and the camera. In our case, this effective time is not . For a field of view, size, (corresponding to about frames) the minimum acquisition time is then 9 min.
Aberrations Correction and Cross-Correlation Spectroscopy
The increase of the efficiency of the nonlinear excitation obtained by the optical aberrations correction, the accuracy in the laser spot positioning, and the use of no movable parts make our setup particularly suited to characterize flows in complex vessel networks,57,58 in confined environments,59 or in vivo8 by means of cross-correlation methods. For these applications, it is essential to be able to select the observation volumes on an image collected with the same setting of the optical setup, particularly for low brightness and signal/noise ratio samples.31 This can be easily achieved by means of our SLM-based microscope as we verify here by studying the flow of microbeads ( size) at decreasing values of the excitation intensity as reported in Fig. 5 for the case of a straight glass microchannel. The amplitude of the cross correlation was to 4 times higher for the aberration corrected CCFs than for the raw data functions [see Figs. 5(b) and 6(c)].
Our spots-array method can be tailored easily to a variety of shapes of the vessels and vessel networks as shown in principle here for a curved microchannel (Fig. 6). The uncertainty in the estimate of the flow direction is determined by the choice of the grid. In particular, for a rectangular grid, the relevant parameter is the angular size with which the upstream spot sees the downstream spot. In Fig. 5(a), this angle (for example between the A and A’ spots) is . The resolution is instead determined by the spacing of the spots along a direction perpendicular to the flow [for example between A and B in Fig. 5(a)], that is . Given the results in Figs. 3(c)–3(e), the interspot distance along a vertical line in Fig. 5(a) cannot be reduced much without decreasing substantially the signal/background ratio. However, one could increase the distance between the vertical lines of spots or shift vertically one column of spots with respect to the other: in both cases, we can easily halve the angular resolution with respect to Fig. 5(a).
Moreover, we can gain additional information on the tracer and on the flow from the decrease of the amplitude of the CCF as a function of the distance between the spots. As it can be seen in Fig. 5(h), the CCF amplitude stays either approximately constant or decreases slightly with this distance. This trend can be due to a low Peclet number60 (i.e., large diffusion coefficients such that ) that would imply an increase in the width of the CCF peak and a parallel decrease of its amplitude. Alternatively, the amplitude of the CCF peak decreases when the flow line is not perfectly aligned with the vector , and therefore, [Eq. (14)]. A simple analysis of Eq. (14) suggests that the amplitude decay should follow an exponential law , where is the angle under which the upstream spot sees the downstream spot (whose size is ). An analysis of the trend of the as a function of provides misalignment angles at most of [for the A’ line in Fig. 5(g)], well within the uncertainty determined by the array geometry. In the cases investigated here, no systematic correlation of the width of the CCF peak with the estimated value was found, indicating that the Peclet number in our case is large (in fact we estimate ).
We notice also that the possibility to create arbitrary patterns of laser spots in the sample would allow us to accurately test the flow field pattern. In the case of the curved microchannel reported in Fig. 6(a) (width and inner radius ), we could have deviations from the axial flow such as Dean flow60 that could be detected in the image plane. However, since the Reynold number in our experiments is relatively low, , any secondary Dean flow60 in the radial direction is within the current uncertainty on the flow velocity direction. In fact, we expect45 a radial component of the velocity , or an angle of misalignment with respect to the laminar flow. In our case, this value , well within the uncertainty due to the grid spacing, . However, as discussed above, this limitation can be overcome by means of an appropriate choice of the spots array.
The time resolution of the flow speed measurements presented here depends on two factors: the concentration of the tracer objects and the uncertainty that we want to reach on the flow speed. We typically acquired traces of 3000 to 4000 frames at a frame rate of 986 Hz for a total acquisition time of about 3 or 4 s. With the particle density of this study, the amplitude of the cross-correlation peak is fairly constant with an error smaller than 10% for total acquisition time . The spread in the CCF amplitude raises to 50% for a total acquisition time . However, the lagtime position of the peak has an uncertainty smaller than 3% for total duration of the order of 0.4 s. Therefore, we can measure the flow speed within a few percent points with a time resolution of 0.4 s. These considerations could vary with microbeads of different brightness and different concentrations.
Finally, we notice that the correction of the aberrations positively affects the uncertainty of the flow speed measurement. For the case of a flow in a straight microchannel [Fig. 5(b)], the width of the CCF peak decreases almost twofold when correcting the aberrations (from to ). The corresponding lagtime does not change appreciably with the application of the corrections but its uncertainty decreases, passing from to . From these values, we can estimate the flow speed (the interspot distance is ) that results , when no aberrations correction is applied, and when the aberrations are corrected, with a twofold reduction of the uncertainty from 0.8% to 0.3%. The width of the CCF peak provides us with an indirect measurement of the beam spot radius through [see Eq. (14)]. The values of the spot size for the two cases are and , for the two cases when we do not correct for the aberrations and when we correct for them, respectively. These figures agree with the measurement reported on fixed grids in Figs. 4(a) and 4(b). The twofold decrease in the speed uncertainty, obtained when applying the aberrations correction, is found also at various speed values [see Fig. 5(e)]: the uncertainty reduces of times in the range . Moreover, the accuracy increases threefold for the absolute speed value: for a nominal flow speed , we find (accuracy ) when the aberrations are corrected, and (accuracy ) when no aberrations correction is applied [Fig. 5(e)].
By means of a purely diffractive microscope based on an SLM, we collected high resolution, aberration corrected two-photon images, on which we positioned with high-accuracy arrays of laser spots and compute CCFs between couples of them. To this purpose, we have combined the shift-and-snap algorithm based on the synchronization of the SLM and the camera with AO minimization of an intensity-based metric. The inclusion of the SLM in the optical path allows also to shed arbitrary light patterns on the sample and to perform cross-correlation measurements over multiple pairs of spots, therefore, expanding the possibility of previous setups.61
In our experience, even for controlled samples in vitro, such as microfluidic devices obtained by PDMS casting, the correction of the aberrations is essential to reach good contrast in the CCFs and to measure accurately the flow velocity. We obtained with the present setup a routine fivefold improvement of the signal/background on the image and an increase in the contrast of the CCF that can reach three times [Figs. 5(b) and 6(c)]. This improvement result in a twofold decrease of the uncertainty in the evaluation of the flow speed and, at least for flow speeds , a threefold increase in the accuracy of the flow speed evaluation.
We expect that the features displayed by the SLM-based correlation microscope can be valuable for the study of hemodynamics in small animal models, such as Zebrafish embryos. In this case, even for transgenic albino embryonic lines, the scattering from the tissue will induce aberrations, possibly anisoplanatic, of the wave front. The possibility of our setup to operate separate aberrations correction over different regions of interest in the field of view will be a valuable help in collecting high-quality dynamic data.
The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.
A small part of this work has been previously published in the conference proceeding “Scanless nonlinear optical microscope for image reconstruction and space-time correlation analysis,” N. Ceffa; F. Radaelli; P. Pozzi; M. Collini; L. Sironi; L. D’alfonso; G. Chirico, Proc. SPIE. 10333, Optical Methods for Inspection, Characterization, and Imaging of Biomaterials III. We acknowledge the support of the funding 2015-ATE-0007 by the University of Milano-Bicocca to G. C. We are grateful to Dr. Paolo Pozzi (TU-Delft, NL) for helpful discussions about the AO algorithms and to Prof. F. Auricchio and Dr. S. Marconi of the university of Pavia, Department of Civil Engineering and Architecture for the 3D printing of the molds.