1.

Introduction

Advances in light sources, computational power, and molecular tags have resulted in a renewed interest in improvements in microscopy. In this work, we focus on four measures of microscope performance: (1) resolution of smaller objects even below the diffraction limit, (2) increased field of view (FOV) toward gigapixel images, (3) the ability to “section” a turbid medium, rejecting out-of-focus objects and thus providing three-dimensional images, and (4) high contrast without exogenous materials, which enables in vivo imaging. For example, in imaging of skin cancers, the clinician would like to resolve subcellular components to at least below the dermoepidermal junction ($\sim 100\text{-}\mathrm{mm}$ deep), with as large an FOV as possible. High-numerical aperture (NA) objectives generally have high magnification, thus limiting the FOV. Confocal reflectance microscopy¹ uses a high-NA objective to provide resolution and sectioning without added contrast, but the high NA and consequent high magnification determine that increased FOV requires mosaics of images.

One relatively new and practical solution to provide quick and accurate sectioning is structured illumination microscopy (SIM).² By projecting a high-frequency modulation pattern through the illumination path onto the specimen, isolation of discrete planes is demonstrated, eliminating clutter that would otherwise obscure useful information. Due to the high frequency of the modulation patterns, it has been found that high-resolution information beyond the diffraction limit, previously inaccessible, can be recovered through innovative processing. Thus, we can reduce the NA and increase FOV while still maintaining most of the resolution and sectioning ability of the confocal microscope. We demonstrate the use of random modulation patterns of incoherent [light-emitting diode (LED)] light for resolution enhancement and sectioning. The optimal solution to form an image with high lateral resolution requires a pattern to be at the Nyquist limit, which, after some reconstruction in the frequency space, allows for double the resolution.³ In practice, there is a trade-off between sectioning depth and the total resolution of the system based on the spatial frequency of the modulation pattern.⁴

Original developments of super-resolution using structured illumination required the precise movements of a sine pattern with a discrete phase and frequency. To illuminate the image fully, a $1/3$ period phase shift must be applied to the sine pattern, requiring an extremely finely tuned mechanical device. Additionally, the pattern must be rotated to a least two other positions to provide resolution enhancement in the $X$- and $Y$-(lateral) directions. Recently, attempts have been made to reduce the necessity for a well-defined sine pattern. Removing this dependency reduces the overall cost and complexity of manufacturing and manipulating the pattern on a micron scale, as well as removing the need for perfect phase alignment within a turbid media.^5–8 Dynamic speckle illumination microscopy uses a speckle illumination pattern to provide sectioning at significant depths but has been demonstrated using fluorescence only.^9,10 Instead, we use a random pattern in reflectance, comprised of $N$ different samples. This method has the benefit of not requiring exact phase shifts but also allows for in vivo imaging without any tagging.^4,11 However, without discrete frequencies and phase shifts, the original method for super-resolution, developed by Gustafsson, cannot be applied to random patterns. Thus, we develop two methods of providing both sectioning at depth and resolution enhancements using a random pattern.

Two algorithms are demonstrated for producing sectioning and super-resolution with an unknown random pattern. The first method we will consider is the maximum a posteriori probability (MAP) processing technique.⁵ MAP processing produces a high-resolution, unsectioned image and a low-resolution, sectioned image, which are then combined to produce a high-resolution, sectioned image. Typically, when MAP is used in conjunction with SIM, a sinusoidal pattern with a known frequency and phase is required to produce sectioning. The literature does not extend the work to sectioning at depth and makes no attempt to demonstrate their functionality against a subsurface object. Additionally, the transfer function of the system must be reasonably characterized to estimate the modulation pattern at the detector accurately. To alleviate some of these constraints, we modify the algorithm to use random patterns that are not known ahead of time. To validate the techniques developed above, we test and consider the quality of sectioning and super-resolution.

Second, we consider a different technique of deblurring that relies on sparse priors.¹² Due to the fact that the point spread function (PSF) will vary as a function of depth, deblurring a thick specimen can often be tedious and a source of errors as it requires multiple estimations of the PSF to correctly reconstruct the image. By sectioning the image using SIM first, we can account for the depth dependency of the PSF. When this sectioned image is deblurred, we show that the results for thick specimens are significantly better than deblurring alone. Sparsity makes the assumption that the object is comprised mostly of sharp edges, allowing us to deconvolve a generic PSF with excellent results. Throughout this paper, we will use the same nomenclature as Levin et al.,¹² referring to this technique as one of sparse priors. To our knowledge, these two techniques described above have not been used for subsurface imaging in biological specimens and provides a very promising technique for extracting both axial and lateral information about a specimen, even within turbid medium.

The deblurring techniques leveraged in this paper fall under a class of algorithms known as deconvolution, where an estimated PSF is decoupled from a signal in the presence of noise.¹³ Each algorithm requires multiple iterations to converge on a solution, which are constrained to their respective regularization functions. To further simplify these methods, we implement all methods using an incoherent light source (reducing cost and increasing safety) and take all images without fluorescent tagging, making extension to most in vivo applications extremely practical. Mainstream techniques typically rely on fluorescent tagging, alleviating many signal-to-noise constraints but limiting the use in in vivo applications. The methods described throughout this paper are applied to images taken in reflectance only. It is understood that less information will be obtained using reflectance. However, given the limited number of available exogenous labels, a label-free technique is of great value. By using a 1951 Air Force Resolution Chart (AFT), we are able to demonstrate the resolution, and sectioning improvements are robust at the surface as well as within a turbid, semiopaque media.

In short, this paper demonstrates an MAP-based algorithm that produces both sectioning and super-resolution without capturing data about the patterns a priori. Further, we show that this MAP processing algorithm benefits from the use of a random pattern for sectioning, as three-phase sectioning does not section well within a turbid media. Second, we use a sparse priors algorithm against a sectioned image. We demonstrate that this provides superior sectioning against targets than deconvolution alone and does not require the adjustment of the PSF for variations in depth, as previous algorithms required.

2.

Data Collection and Processing

2.2.

Experimental Setup

Our experimental setup is specifically designed to allow for manipulation of the pupil on both the transmit (before the specimen) and receive (after the specimen) side of the illumination path. Knowing that the pupil plane contains the Fourier transform of the image, a simple aperture can be used in each pupil to band limit the spatial frequency of the microscope.¹⁶ Constricting the aperture changes, the diffraction limit of the system. This adjustment allows for the raw, high-NA images to be compared to the processed low NA counterpart. The following sections will show how each processing technique is conducted and how well they compare to the ground truth.

In all experiments, 80 random, unknown patterns are used to thoroughly illuminate the entire specimen. The random patterns are binary in nature using the rand() function in MATLAB^® with uniform distribution and adjusted to a 25% fill factor. Using a random pattern, the frequency of the pattern spans the entire bandwidth of the system and is limited only by $\mathrm{NA}=0.25$. This fill factor was selected to ensure that the specimen was adequately illuminated and that we were able to achieve strong contrast at depth. This contrast will be demonstrated in Sec. 2.2 as we show objects can still be resolved even through a semiopaque media. Using a lower fill factor allows us to achieve deeper sectioning but requires additional images.¹⁷ The number of samples used, 80, was selected empirically. There is a trade-off between image contrast and processing time, which both increase as the number of images increases. Previous work has shown that in vivo sectioning is possible in as little as 30 frames.⁴ However, here, the higher number of samples is chosen to best demonstrate the processes developed in this paper.

A 635-nm incoherent LED with a linewidth of 17 nm was used as the light source. The patterns were projected onto the specimen using a TI Lightcrafter digital micromirror device (DMD). The images were then captured using an Allied Vision Guppy PRO F-125 CCD camera with a sampling rate of 22 Hz, providing all 80 samples in just over 3.5 s. All data were processed in MATLAB^® R2013a, using an Intel Core i7-2675QM CPU at 2.20 GHz. The objective has a nominal magnification of $10\times$ in air with an NA of 0.25. During the experimentation, by adjusting the apertures in the pupil planes, the NA of the system was significantly lowered to test for super-resolution. A layout of the microscope is shown in Fig. 1. Before constricting the apertures, the maximum resolution of the system was measured as $\sim 1.8\mu \mathrm{m}$ in agreement with Zemax simulations of the system. All images are compared to their wide-field equivalent that is constructed by taking the average of all images as described by

Eq. (3)

$$I_{\mathrm{WF}}(x,y)=\frac{1}{N}\sum _{n=1}^N{I}_n(x,y).$$

This sum represents the image that one would expect to see through the eyepiece of the microscope without any additional processing or modulation. Figure 1 provides a graphical representation of the microscope. For the structured illumination processing to work properly, the pattern projected by the DMD must be conjugated with the focal plane, as well as detection plane. The dashed lines in the image planes (labeled with an “$I$”) denote the location of the pattern in space. All other sections labeled with an “$F$” denote the pupil plane, which contains the Fourier transform of the image. In $F1$ and $F4$, an aperture is added to low-pass filter the frequency content of both pattern and specimen before it arrives at the camera.

Fig. 1

Layout of the microscope. Each image plane is labeled with an “$I$” and each pupil plane is labeled with an “$F$.” A beam splitter is located just before the objective lens to relay the light toward the CCD device. Placing an aperture in $F1$ and $F4$ gives the ability to artificially lower the NA of the system.

2.3.

Sectioning Techniques

Many different sectioning techniques have been developed to provide sectioning from three phases.^8,18–21 However, as shown by Hoffman and DiMarzio,¹¹ the susceptibility of phase misalignment in turbid media renders most images useless. Using a random pattern, rather than one of discrete phase changes, ensures that sectioning at depth produces nominally artifact-free images.⁴ To decouple the modulated and unmodulated portions of light, the following pair-wise comparison [Eq. (4)] is applied and results in uniformly sectioned images when using multiple, random, modulated patterns. This is just an extension of the typical differencing scheme from Eq. (2) but ensures that all 80 images are subtracted from one another

Eq. (4)

$$I_{\sec }(x,y)={\left\{\sum _{j=1}^{N-1}\sum _{k=j+1}^N{[I_j(x,y)-I_k(x,y)]}^2\right\}}^{1/2}.$$

Additionally, we rely heavily on the use of a random modulation pattern, as this ensures that we are working at the limits of the OTF. The use of random patterns increases the amount of data required for sectioning but simplifies the hardware and characterization of the system required for proper functioning. An experiment is run to quantify the axial resolution of this sectioning technique. The random pattern is projected onto a mirror at a focal plane where the pattern is completely in focus. The location of the mirror relative to the focal plane is adjusted by increments of $0.5\mu \mathrm{m}$. The results are plotted in Fig. 2. At $1.0\mu \mathrm{m}$, there is a drop in amplitude across all frequencies of about 1 to 4 dB, enough to blur the pattern, but not likely to significantly impact the sectioning. However, around $1.5\mu \mathrm{m}$ difference from the focal plane, the amplitude drops by 5 to 15 dB throughout the spectrum. With this much loss in signal, there is no longer a detectable signal from the modulation pattern, thus, all light from these depths will be rejected by our sectioning algorithm. From this, we should expect a strong drop in signal outside of $\pm 1.5\mu \mathrm{m}$ (above and below the focal plane) yielding a sectioning thickness of about $3.0\mu \mathrm{m}$.

Fig. 2

Comparing the signal strength of the pattern at various distances from the focal plane. The same pattern is projected on a mirror at different relative distances. As the distance increases, the strength drops off significantly.

Another advantage of a random broadband pattern is that the maximum frequency present will always be limited by the OTF of the system. This ensures that the image will always be modulated at the Nyquist limit of the system. This can be visualized in Fig. 3. Here, we show experimentally the frequency spectrum of three patterns projected onto a mirror. Figure 3(a) contains a low-frequency discrete square pattern well within the limits of the OTF (first-order side bands are shown with white arrows). Because this is a square wave, we note the higher order harmonics make up the diagonal line in the figure. The black circle in the figure represents the approximate OTF of this system. Figure 3(b) is a high-frequency square pattern much closer to the edge of the OTF. This setup will provide much better resolution enhancement during reconstruction. As demonstrated by Gustafsson,³ a total of $2\times$ resolution improvement can be obtained when the frequency of the pattern exists at the edge of the OTF. Should the OTF shrink, this may begin to alias, no longer providing sectioning or super-resolution. However, the random pattern in Fig. 3(c) conforms to the OTF continuing to provide both sectioning and super-resolution at any OTF size. This feature obviates the need to characterize the system before developing the projection patterns. Ultimately, there is a trade-off between resolution and depth of sectioning. For a high-resolution modulation pattern, we are able to get close to the full $2\times$ improvement as demonstrated by Gustafsson. However, within a turbid media, contrast is rapidly lost, producing poor sectioning. In contrast, a low-frequency pattern provides much better sectioning at depth but limited resolution improvement.

Fig. 3

Comparison of three different modulation patterns, magnitude in dB. Aperture stop superimposed with a black circle. (a) Low-frequency square wave pattern projected on top of mirror target. First-order harmonics (arrows in white) is within aperture stop providing sectioning at depth but limited resolution improvement. (b) High-frequency square wave pattern projected on a mirror closer to the limits of the spatial resolution of the system, providing limited sectioning but increased resolution improvement. (c) Random modulation pattern on a mirror, spanning the entire allowable frequency space, providing both sectioning and resolution improvement at the cost of additional data samples.

2.4.

Super-Resolution Techniques

2.4.1.

Maximum a posteriori probability

The first technique that we explore is known as MAP processing. The MAP processing algorithm is outlined in Fig. 4. Previous implementations of MAP processing required the pattern to be known a priori; however, our research will show that we can still perform sectioning and super-resolution, even without prior knowledge of the patterns. Starting with 80 samples modulated with a random pattern [$I_n(x,y)$], a wide-field image, $I_{\mathrm{WF}}$, is produced using Eq. (3), and a sectioned image, $I_{\sec }(x,y)$, is produced using Eq. (4). The wide-field image is then subtracted from each sample to produce an estimate of the random modulation pattern as follows, $M_n^{*}(x,y)=I_n(x,y)-I_{\mathrm{WF}}(x,y)$. This operation is completed in the image domain and ideally removes the specimen component of the image ($I_{\mathrm{WF}}$) from the pattern + image, returning on an estimation of the pattern only. The results from this process can be seen in Fig. 5. The MAP processing then attempts to minimize the mean square error between the estimated high-resolution images and the observed images; $I_n(x,y)$ and $M_n^{*}(x,y)$, finding the most probable high-resolution image required to produce the sampled array of low-resolution images. As described by Lukeš et al.,⁵ a gradient decent algorithm is applied to iteratively minimize the following equation:

Eq. (5)

$$I_{\mathrm{HR}}=\arg {\min }_x[\sum _{n=1}^N{\Vert I_{WF}-h\otimes M_n^{*}x\Vert }^2+\lambda \mathrm{\Gamma }(x)].$$

Here, we are solving for the high-resolution image $I_{\mathrm{HR}}$ from the constructed image in Eq. (1), where $\lambda \mathrm{\Gamma }(x)$ is a regularization function, composed of first-order derivatives at each pixel. To minimize noise in the image, MAP processing applies Gaussian priors, slightly smoothing out the image. From this, a high-resolution, unsectioned image is produced. This high-resolution is first high-pass filtered and then combined with a low-passed version of the sectioned image. By combining the two in the frequency domain, a high-resolution, sectioned image is produced.^21,22 This process is completed by extracting the low- and high-frequency components of their respective images and adding the two together as described by

Eq. (6)

$$I_{\mathrm{HR}\text{-}\mathrm{SIM}}={\mathcal{F}}^{-1}\{(1-\beta )\mathcal{F}\{I_{\sec }\}\exp (-\frac{f^2}{2{\rho }^2})+\beta \mathcal{F}\{I_{\mathrm{HR}}\}[1-\exp (-\frac{f^2}{2{\rho }^2})\left]\right\},$$

where $I_{\sec }$ is spectrally filtered by a Gaussian function of standard deviation $\rho$ at frequency $f$, weighted by $\beta$ and combined with an inversely filtered version of $I_{\mathrm{HR}}$. The two filtered images are combined in the frequency domain, which after taking the inverse Fourier transform (${\mathcal{F}}^{-1}$) produces the final high-resolution, sectioned image $I_{\mathrm{HR}\text{-}\mathrm{SIM}}$.

Fig. 4

Flow chart outlining the MAP processing.

Fig. 5

For a single frame captured, which is comprised of the specimen modulated by the random pattern, above demonstrates the isolation of the pattern from the specimen (here, the target is an air force resolution chart). This is achieved by subtracting (a) the wide-field image producing an estimation of the pattern, as shown in (b). White bar represents a distance of $50\mu \mathrm{m}$.

Typical implementations of MAP expect that the modulation pattern is known a priori.⁸ In the case of imaging in turbid media, the exact position of the pattern will not be consistent from specimen to specimen. Thus, to remove this constraint, we estimate the modulation after the data have been collected. This removes the necessity of characterizing the system ahead of time, whose positions will change within turbid media, as required in previous implementations of MAP. The MAP processing algorithm that is implemented throughout the rest of this paper is based on a modified version of the SIMToolbox, developed by the Multimedia Technology Group, Czech Technical University, Prague.²² The algorithm has been modified using the pattern estimation technique described above. On average, the MAP processing took $\sim 5$ iterations to converge on a solution in a total time of 35 s. We note that all experiments going forward will use our modified version of the MAP processing algorithm, which allows for the use of blind modulation patterns. All experiments use no a priori knowledge of the pattern and are integrated into the MAP algorithm. The above equations describe the modifications applied in this paper, but we refer readers to the original paper for additional details.

2.4.2.

Sparse priors

Next, we apply a deconvolution technique that relies on sparse priors. In sparse reconstruction, images are assumed to be mostly binary in nature and their details of interest are expected to have sharp edges. These assumptions allow for high-quality reconstruction against small point scatters and thin lines, such as fibers. Objects such as these, which are beyond the working diffraction limit, typically reconstruct nicely. The deconvolution processing is based on the algorithm developed by Levin et al.¹² and implemented as described. To estimate the super-resolution image, we minimize the following:

Eq. (7)

$$I_{\mathrm{HR}\text{-}\mathrm{SIM}}=\arg {\min }_x|h\otimes x-I_{\sec }|+\sum _{ij}\rho [x(i,j)-x(i+1,j)]+\rho [x(i,j)-x(i,j+1)],$$

where $\rho$ is a heavy tailed function, where $\rho (z)=|z{|}^{0.8}$ while iteratively solving for $x$, which represents an intermediary, sharp, image until convergence is achieved.¹² In previous implementations, this algorithm relied on varying the PSF to account for depth. Our research shows that, with sectioning, this depth dependency can be removed, requiring only a single PSF estimation for deconvolving the image. Additionally, within turbid media, the deconvolution process is far less effective without sectioning first.

The algorithm is outlined in Fig. 6. After the 80 modulated images are collected, the sectioned image is produced. The sectioned image is then deconvolved from an estimated PSF. The PSF is iteratively refined until the algorithm converges. An assumption of sparse priors is applied in this technique. There are two reasons why sectioning first is necessary in this algorithm: first, the PSF will vary as a function of depth. By sectioning first (isolating a single depth plane), the PSF should be approximately uniform across the entire image, greatly simplifying the deconvolution procedure. Second, the use of sparse priors will constrain the edges of single point objects. In our case, the modulation pattern itself contains many point objects. By sectioning first, the pattern is nominally removed, leaving only the specimen to be processed. Similar to the MAP processing, this algorithm is run iteratively until convergence, continuously updating the PSF. For our application, we run this process about 200 times in 28 s, including sectioning, which provides a reasonable trade-off between overall convergence and minimal processing time. Adjustment of the total number of iterations or increasing the size of the PSF will have a dramatic effect on how long it takes to complete the processing. The process has been shown to work well in other microscopy techniques in various biological specimens.^23–25

Fig. 6

Flow chart outlining the sparse priors processing.

For this experiment, a PSF is modeled as an Airy disk²⁶ and then the Fourier transform is taken to produce the OTF estimation

Eq. (8)

$$\mathrm{OTF}(f)=\frac{1}{\pi }\left\{2{\cos }^{-1}\right(\frac{|f|}{f_c})-\sin [2{\cos }^{-1}\left(\frac{|f|}{f_c}\right)\left]\right\},$$

where $f_c$ is the cutoff frequency of the OTF. For this paper, all PSF values are selected empirically to get the best results. However, this technique can be improved by creating a smarter selection criterion for the PSF, producing higher quality images and faster convergence time. For these experiments, the amount of blur for any given pixel is modeled by a symmetrical 2-D function that smears any single pixel across 12 pixels in the $x$- and $y$-directions, resulting from Eq. (8). Again, the reader is referred to the original paper by Levin for details about the algorithm that go beyond our modifications.

3.

Results

A 1951 USAF resolution test chart (AFT) is used to evaluate and compare the two algorithms. For the experiments going forward, we will define the resolution as the distance between two black lines on the chart, regardless of the thickness of the line. Figure 7 compares the full-resolution image at full NA, 0.25 (a), to low-resolution image when the aperture in the detection path is constricted (b). At full resolution, the elements can be resolved, with the smallest lines $\sim 2.2\mu \mathrm{m}$ in width (group 7, element 6). The aperture is then closed making the smallest resolvable line $\sim 6.2\mu \mathrm{m}$ in width (group 6, element 3), corresponding to an NA of $\sim 0.07$. It is found that both the MAP (c) and sparse processing (d) provide a resolution improvement. The MAP processing reveals elements with a linewidth of $4.4\mu \mathrm{m}$ (group 6, element 6), increasing the resolution by $\sim 1.4\times$. The sparse prior processing reveals elements with a linewidth of $\sim 3.9\mu \mathrm{m}$ (group, 7 element 1), increasing the resolution by about $\sim 1.6\times$. With respect to total resolution improvement, it can be seen that the sparse processing produced an image that more accurately reflected the pattern seen in the ground truth image (a). However, it is found that both techniques introduce a few artifacts in very low-frequency regions. These artifacts can be seen in the background, as well as the black box between the 7 and 6. The cause of these artifacts is likely nonuniform illumination from the random patterns that are used. These artifacts can also be seen in the wide-field image (b), where there is still some structure present in the image. Further, we see that relative to the MAP processing, the sparse deconvolution overregularizes the location of the three lines, reconstructing the lines as slightly thinner than those seen in the ground truth image. Cross sections from two elements of the AFT have been plotted in Fig. 8(a); group 6 element 5 and Fig. 8(b); group 7, element 1. Here, it is clear how well the three elements are resolved, as well as the contrast improvement provided by each of the techniques.

Fig. 7

Comparison of the AFT before and after processing, using 80 random patterns. Here, we give an “effective NA” where we calculate the approximate NA based on the smallest resolvable object. (a) The full-resolution target as a ground truth, with the aperture in the pupil fully dilated ($\mathrm{NA}=0.25$, resolution $\sim 2\mu \mathrm{m}$: group 7, element 6), (b) the resolution target after the aperture has been closed ($\mathrm{NA}=0.07$, resolution $\sim 5.5\mu \mathrm{m}$: group 6 element 3), (c) after MAP processing (effective $\mathrm{NA}\sim 0.1$, resolution $\sim 3.9\mu \mathrm{m}$: group 6, element 6), and (d) after sparse processing (effective $\mathrm{NA}\sim 0.13$, resolution $\sim 3.1\mu \mathrm{m}$: group 7, element 1). Black scale bar in bottom right corner of each image represents a distance of $50\mu \mathrm{m}$.

Fig. 8

(a) Cross section of the AFT from group 6, element 5, measuring a resolution of $4.92\mu \mathrm{m}$. Both processing techniques match the ground truth but are otherwise lost in the wide-field image. (b) Cross section of the AFT from group 7, element 1, measuring a resolution of $3.91\mu \mathrm{m}$. Only the deconvolution with sparse priors matches the ground truth. Each cross section is from a single line of pixels from the image.

Next, microbeads, made of polystyrene, $\sim 2\mathrm{mm}$ in diameter are placed on top of a mirror, as shown in Fig. 9. The resolution of the system is reduced ($\mathrm{NA}\approx 0.13$) and the wide-field image is shown in Fig. 9(a). The region boxed in red is shown in the image below [Fig. 9(d)]. The MAP processed image is shown in (b) and (e), whereas the sparse priors processed image is shown in (c) and (f). In the low-resolution image, individual beads cannot be discerned when they are adjacent to one another. The MAP processing creates contrast and sharpness around the edges. However, because the MAP processing uses a Gaussian prior as its regularization function, it has a tendency to smooth over edges making many beads appear as one large blob. Although it sharpens large areas of contiguous beads, it is incapable of segregating adjacent beads from one another. In the sparse priors technique, the processing does an excellent job of differentiating each bead. Arrows have been superimposed onto the figure, to show two regions where the resolution has undergone marked improvement. In Fig. 10, a cross section of the beads shows an almost a $2\times$ resolution improvement from the original image to that of the sparse priors processing. We measure the full width half max (FWHM) of the low-resolution image to be $\sim 4\mu \mathrm{m}$ as expected. After the sparse processing, there are two well-defined beads with a FWHM of $2\mu \mathrm{m}$, demonstrating the accuracy of this procedure. As for the MAP processing, it is apparent that the deconvolution process is merging the individual beads in the image making it difficult to isolate one from one another.

Fig. 9

Images taken of $2\text{-}\mu \mathrm{m}$ microbeads on a mirror (beads are in black), showing the differences between each of the processing techniques. (a) The wide-field image where the individual beads cannot be seen. The red box zooms in on the region shown in the image below (d). (b) and (e) The MAP processing and zoomed-in region, respectively. (c) and (f) The sparse processing and the zoomed-in region, respectively.

Fig. 10

Cross section of two adjacent beads, taken from Fig. 4.3. Amplitude has been inverted for clarity. Using the sparse processing, the two beads are now distinguishable from one another. The MAP processing is incapable of isolating the two beads from one another.

Given the desire to apply these techniques to in vivo applications, a biological specimen is investigated to ensure that both the super-resolution and sectioning aspects are present in the reconstructed image. Figure 11 shows a layer of onion cells placed $\sim 30\mu \mathrm{m}$ above the AFT. The space in between the two targets is filled with ultrasound gel ($n=1.33$). Based on previous experiments on our sectioning technique, we have shown that sectioning resolution of this random pattern technique is $\sim 1.0\mu \mathrm{m}$. Therefore, we expected to remove scattered light from the AFT at depth, as well as scattered light from the onion cells a few microns below the surface. In the first image (a), dark areas are seen below the onion cells due to absorption of the light from AFT beneath the cells. Additionally, contrast among each of the cells is reduced from scattered light throughout the sample. The sectioning process is verified by noting that the dark regions have been removed in both (c) and (d), as well as clarifying the interface between cells. In both of the processed images, distinct lines between the cells of each pair are seen, giving strong contrast in the image. Comparing the MAP processing in (c) to the sparse processing in (d), many of the point scattering objects, as well as the regions between the cells are even further defined. To ensure that the sectioning and super-resolution are complimentary, the deconvolution is applied directly to the wide-field image (b). Without applying the sectioning first, the scattered light from the onion cells and AFT below are not rejected. As such, the super-resolution technique actually enhances the contrast of the AFT below. It is worth noting the contrast at the edges of the onion cells. In the wide-field example, the regions among cells are almost indistinguishable. Both processing techniques increase the contrast greatly. In reviewing the sparse priors technique (d), the edges are further narrowed, producing high-contrast, high-frequency edges between each of the cells. In the deconvolution only image, some of the lines are enhanced slightly, however, they are not nearly as effective without the sectioning.

Fig. 11

Images taken of onion cells: (a) plots the wide-field image of the cells, the AFT can be seen beneath cells. (b) The wide-field image after sparse deconvolution (no sectioning), (c) contains the onion cells after using the MAP processing, and (d) contains the onion cells after the sparse processing.

To further verify the technique, the focus is adjusted to a depth below the onion cells, as demonstrated in Fig. 12. The lines are distorted as a result of an index of refraction change from the air objective to the onion cells. Use of an index matching material would likely mitigate this distortion, greatly improving the results. It can be seen that the sparse priors technique gives additional, albeit, modest resolution improvement, which is consistent with previous results. Specifically, three lines in group 8 element 6 (lower-right side) are now resolvable, which are not visible in the other three images. There are some air bubbles between the onion cells and the resolution chart, which is present in the ultrasound gel. Due to the index of refraction change, in some areas, the modulation pattern does not focus on the correct plane. As a result, there are some dark lines and spots in the image. However, on the whole, the sectioning has removed much of the scattered light from the cells above the resolution chart, providing better contrast.

Fig. 12

Images taken of the resolution chart $\sim 30\mu \mathrm{m}$ beneath the onions: (a) contains the wide-field image of the AFT beneath the onion cells, (b) the wide-field image after sparse deconvolution (no sectioning), (c) contains the AFT after using the MAP processing, and (d) contains the AFT after the sparse processing.

Last, we compare these methods to two other common deconvolution techniques without any sectioning. We apply a Lucy–Richardson and maximum likelihood deconvolution algorithm. Each method was implemented in MATLAB^® 2013a by MathWorks, using a template PSF, similar to the one applied in the sparse deconvolution method. These results are plotted in Fig. 13. This plot looks at cross sections from the resolution bars of Fig. 12, group 6, element 6 (bottom-right section of the imaged resolution chart). The plot on the left side of Fig. 13 plots a cross section of the vertical bars. Both the sparse method and MAP processing methods greatly increase the contrast of these bars. The other two methods produce results similar to no processing at all. The right graph plots the cross section of the horizontal bars. The sparse method succeeds in producing contrast between the lines while the MAP processing improvement is modest at best. However, the other two deconvolution methods, without sectioning, again do not appear to produce quality results. This experiment agrees with the original hypothesis that applying the sectioning first allows for superior deconvolution and better overall results.

Fig. 13

Comparison of MAP and sparse methods against Lucy–Richardson and maximum likelihood method. These plots compare the resolution bars of group 6, element 6 ($4.38\mu \mathrm{m}$). In (a) cross section of the vertical bars is plotted. In (b) cross section of the horizontal bars is plotted.

4.

Conclusion

Random patterns of incoherent light can be used in SIM to provide resolution enhancement and sectioning. These experiments demonstrate that even with limited a priori knowledge of the pattern and the optical system we can, with the right processing, achieve both optical sectioning and spatial resolution beyond the diffraction limit. Because the algorithm is robust with respect to variations in the pattern, the severe alignment constraints of many structured illumination approaches are eliminated. Given that traditional three-phase SIM and super-resolution SIM do not work at depth, we are able to show that random patterns are capable of providing both. Recognizing that only a few fluorescent probes are approved for in vivo imaging, we have chosen to test our system and algorithms with reflectance imaging, which requires no exogenous materials. In this work we achieved a transverse resolution enhancement of a factor of 1.4 using the MAP algorithm with Gaussian priors and 1.6 with sparse priors.

Considering the particular application of imaging skin, to obtain results comparable to conventional laboratory examination of hemotoxylin-and-eosin-stained biopsy specimens, we need resolution of about 1 mm to match the laboratory microscope resolution and sectioning of about 5 mm to mimic the typical microtome sections. Using this approach to SIM, we can reduce the NA and magnification of the objective to improve the FOV while recovering much of the resolution and sectioning ability.

The sparse priors approach is particularly suited to isolated point objects, such as the subcellular organelles, and provides resolution enhancement of a factor of 1.6, even without knowledge of the random patterns or the PSF of the microscope. For less sparse objects, the Gaussian priors algorithm provides resolution enhancement of a factor of 1.4. We have demonstrated a particularly useful technique for producing high-quality sectioned images at resolutions and FOVs greater than conventional microscopy, with minimal knowledge about the system a priori. This advance should prove to be extremely useful in the context of low-cost, low-maintenance microscopy in in vivo applications.