1.

Introduction

In recent years, there have been numerous advances in the development and use of upconverting lanthanide nanoparticles (UNPs) for biomedical imaging.^1,2 These efforts are driven by the increasing need for improved contrast agents and robust sensors for advanced imaging modalities such as multiphoton and super-resolution microscopy.^3,4 Of particular interest are ${\mathrm{NaYF}}_4$ nanocrystals, chosen for their low phonon energy, codoped with a combination of trivalent lanthanide ions, commonly ${\mathrm{Yb}}^{3+}$ and either ${\mathrm{Er}}^{3+}$ or ${\mathrm{Tm}}^{3+}$.⁵ UNPs have numerous advantages over more traditionally used molecular fluorophores. Excitation of these particles typically takes advantage of the ${}^2{\mathrm{F}}_{7/2}\to {}^2{\mathrm{F}}_{5/2}$ transition in ${\mathrm{Yb}}^{3+}$ ions at 980 nm. Energy transfer from these sensitizers to other nearby ${\mathrm{Ln}}^{3+}$ ions can cause multiple excitation steps, culminating in anti-Stokes shifted emission. As a result, autofluorescence, a significant contributor to unwanted background signal in fluorescence imaging, is absent. In addition, UNPs are extremely photostable, and exhibit almost no loss in luminescence after long periods of illumination except at very high excitation powers.^6,7 Energy transfer upconversion (ETU), the mechanism by which these particles are excited, is very efficient when compared to other multiphoton excitation processes, in part because it makes use of real intermediate electronic states during excitation. As an example, ETU is as much as ten orders of magnitude more efficient than two-photon absorption without a real intermediate electronic state in similar materials (Fig. 1).⁸

Fig. 1

Diagram of ETU and two-photon absorption with efficiencies for each process in lanthanide materials.⁸ Solid arrows represent absorption and emission of light. Dashed arrows represent nonradiative population and depopulation processes that are caused by energy transfer between two lanthanide ions. The wavy arrow represents internal conversion.

These properties make a compelling case for the use of UNPs as exogenous contrast agents for two-photon scanning microscopy. The relationship between excitation power and emission intensity is still quadratic for two-photon ETU, leading to optical sectioning similar to two-photon microscopy with endogenous fluorophores.⁹ While ETU is very efficient when compared to other forms of multiphoton excitation, it is still considerably less efficient at producing light than single photon processes. Quantum yields for ${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}({\mathrm{Tm}}^{3+})$ are typically between 0.005% and 3%.¹⁰ The electronic transitions that result in lanthanide luminescence typically occur within the 4f orbital of these ions, and are consequently forbidden. An immediate consequence of this property is that the absorption cross-sections of lanthanide ions are relatively small. The highly sensitive detection obtained with two-photon scanning microscopy techniques helps to mitigate these properties.

Another consequence of these luminescent transitions being forbidden is that the luminescence lifetime of excited UNPs is long; lifetimes greater than 100 μs are not uncommon. This has advantages in some applications, particularly those that sense changes in the local environment by measuring changes in the luminescence lifetime.^11–15 However, this long lifetime is a hindrance when imaging with a fast scanning microscopy technique, with a pixel dwell time as short as 1 μs. For a pixel dwell time less than 500 μs, nanoparticle emission is streaked along the fast-scan axis of the microscope (Fig. 2). This streaking is the result of nanoparticle emission from previously scanned locations being attributed to the current location of the excitation focus. In addition to emission streaking, the long lifetime of ${\mathrm{Ln}}^{3+}$ excited states makes it fairly easy to saturate the first step of the ETU excitation process. This saturation results in a change in the relationship between emission intensity and excitation power, from quadratic to linear, for many of the emission lines in UNPs.⁷ In essence, excitation of UNPs begins to behave like a single photon process and optical sectioning is lost.

Fig. 2

A demonstration of the streaking observed by two-photon scanning microscopy when contrast agents with long luminescence lifetimes are used. The image is of 665 nm emission from lipid coated ${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}$ nanoparticles following 980 nm excitation. The fast-scan axis represents the direction of a single line acquisition, with lines added sequentially in the direction of the slow scan axis. Long-lasting luminescence collected in nondescanning mode obscures the actual distribution of luminescent nanoparticles. In this example, the pixel dwell time was 4.6 μs per pixel and the luminescence lifetime is approximately 358 μs.

These now recognized characteristics of UNPs obscure their potential as robust and efficient contrast agents for advanced imaging modalities based on laser scanning. Both Li’s and van Veggel’s groups have reported on these issues previously, and each has developed a working solution to mitigate some of the effects of a long luminescence lifetime at the expense of, respectively, light throughput and optical sectioning.^16,17 In addition, Faris’s group has addressed a similar issue in multiphoton imaging with europium chelates.¹⁸ In spite of these efforts, applications of UNPs in biomedical imaging remain limited, with the bulk of data generated to date obtained by wide field imaging techniques.^1,6

Herein, we report an image processing method for mitigating the effects of this long luminescence lifetime on lateral resolution while preserving a short pixel dwell time and low excitation power, both of which are essential for rapid acquisition of luminescence images while maintaining optical sectioning. This method employs Richardson-Lucy (R-L) deconvolution by the time resolved luminescence of UNP emission in order to remove pixel streaking, resulting in images with roughly equivalent resolution in the fast-scan and slow-scan axes of a two-photon scanning microscopy image. As proof of principle for the application of deconvolution to this specific problem, ${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}$ nanoparticles were synthesized by thermal decomposition, rendered water compatible by lipid coating, and functionalized with an antibody for epidermal growth factor receptor (anti-EGFR). Following functionalization, these UNPs were incubated with 12-μm-thick sections of mouse ear tissue, which is known to express EGFR, and imaged using a two-photon scanning microscopy system.

2.

Methods/Materials

3.

Results and Discussion

3.1.

Nanoparticle Characterization

Following synthesis, ${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}$ nanoparticles had an average number weighted diameter of approximately 60 nm. Once lipid coated, these UNPs had an average diameter of $\sim 60\mathrm{nm}$ and were easily dispersed in water, indicating that the bulk of the nanoparticles were coated by a single monolayer. The size distribution had a full width half max of 11.5 nm. While an increase of $\sim 4\mathrm{nm}$ in diameter is expected following the addition of a lipid monolayer, this change was not detectable with the particle sizer used. Following conjugation to anti-EGFR and subsequent purification of the sample by dialysis, a UV absorption peak at $\sim 280\mathrm{nm}$ was observed, verifying the presence of anti-EGFR attached to nanoparticles. Luminescence decay profiles were collected for uncoated and coated particles. The red emission from uncoated particles in dichloromethane was fit by the exponential rise and decay described previously with a rise constant of 75.2 μs and a decay constant of 348.9 μs. This fit had a reduced ${\chi }^2$ of 1.62. Fitting the luminescent decay of red emission from the lipid coated UNPs to the same equation resulted in a rise constant of 23.8 μs and a decay constant of 358.3 μs, with a reduced ${\chi }^2$ of 1.11 (Fig. 3).

Fig. 3

Time-resolved luminescence of ${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}$ nanoparticles following excitation by a 3 ns laser pulse at 980 nm. (a) Rise and decay of red emission from uncoated particles in dichloromethane. A bin width of 2.048 μs was used. (b) Rise and decay of red emission from lipid coated particles in MES. A bin width of 4.096 μs was used.

3.2.

Optical Sectioning with UNPs

In order to determine an excitation power density that would maximize emission intensity while retaining optical sectioning, we investigated the effect excitation power density has on axial resolution. For the measurements shown in Fig. 4, a dwell time of 19.6 μs per pixel was used. As can be seen in Fig. 4, as excitation power density is decreased from $490\mathrm{kW}/{\mathrm{cm}}^2$ to $850\mathrm{W}/{\mathrm{cm}}^2$, the axial resolution improves from a full width half maximum (FWHM) of 37 μm to approximately 5 μm. Using this information, we chose an excitation power density of approximately $21\mathrm{kW}/{\mathrm{cm}}^2$ for our imaging experiments, which corresponds with an axial FWHM of 7 μm.

Fig. 4

Axial resolution versus excitation power. Axial resolution was determined by imaging emission from a single, lipid coated nanoparticle at 0.5 μm steps for 40 μm along the $z$-axis. Images used were 255 by 255 pixels, and were taken with a 19.6 μs dwell time. The field of view used was 100 μm by 100 μm. Excitation power was calculated assuming a 1 μm radius spot size. The axial FWHM of the system was measured as $\sim 2.5\mu \text{m}$.

3.3.

Richardson-Lucy Deconvolution

The images in Fig. 5 were collected using a scan speed of 200 lines per second, a 510 by 510 pixel resolution, and a 400 μm field of view. The resulting dwell time was 9.23 μs per pixel. The image shown in Fig. 5(a) was obtained by summing eight images acquired at this scan speed, resulting in a total image acquisition time of 18.7 s. Due to the long luminescence decay time of nanoparticle emission at 665 nm, the resulting luminescence image is heavily streaked along the fast-scan axis of the image. This streaked image can be represented as the convolution of the true image with the exponential rise and decay of UNP luminescence,

Eq. (1)

$$g(x,y)=f(x,y)\otimes h(x,y)+n(x,y),$$

where $g$ is the blurred luminescence image, $f$ is the true image that we wish to resolve, $\otimes$ is the convolution operator, $n$ is noise, $x$ and $y$ are positions along the slow-scan and fast-scan axes respectively, and the PSF is described by

Eq. (2)

$$h(x,y)=(-{\mathrm{Ae}}^{\frac{-y}{v\tau 1}}+{\mathrm{Be}}^{\frac{-y}{v\tau 2}})\int \delta (x)\mathrm{d}x,$$

where $\delta (x)$ is a Dirac delta along the slow-scan axis and $v$ is the fast-scan speed. In trying to solve this equation for $f$, we are fortunate because we can easily determine the parameters of this point spread function (PSF) with single photon counting. However, even with a known PSF, it is difficult to resolve the true image using a simple linear inverse filter due to the presence of significant photon noise, especially at the tail of nanoparticle emission. Instead, we apply an iterative deconvolution technique to retrieve our desired image. R-L deconvolution is an iterative deconvolution technique that is well known in the astronomy and medical imaging communities and is commonly used to resolve blurred images generated in the presence of significant Poisson distributed noise. R-L deconvolution is described by

Eq. (3)

$${\hat{f}}_{(k+1)}={\hat{f}}_{(k)}(h*\frac{g}{h\otimes {\hat{f}}_{(k)}}),$$

where $f_{(k)}$ is the estimated image after $k$ iterations and ∗ is the correlation operator.²⁵ This iterative method almost always converges; however, it is usually unwise to iterate to convergence, as the deconvolution begins to amplify noise after many iterations.

Fig. 5

A 12-μm section of murine ear stained with anti-EGFR-${\mathrm{NaYF}}_4:{\mathrm{Yb}}^{3+}$, ${\mathrm{Er}}^{3+}$ nanoparticles, false colored green here. The grayscale images shown are tissue autofluorescence at 460 nm following 780 nm excitation. UNP emission was collected at 665 nm following 980 nm excitation. Images were 510 by 510 pixels and represent a 400 μm field of view. Dwell time was 9.2 μs per pixel. The excitation power density was $21\mathrm{kW}/{\mathrm{cm}}^2$ for the upconversion luminescence image and $\sim 10\mathrm{MW}/{\mathrm{cm}}^2$ for the autofluorescence image. (a) Full image before deconvolution. The arrows at top and bottom represent the location of the profile shown in (b). (b) Profile taken along a single scan line of an image of upconverting nanoparticles bound to tissue. The dashed line is equivalent to the PSF used for deconvolution, shifted and scaled to overlap with the rise and decay of a single particle. PSF amplitudes were set equal for simplicity. (c) The same profile following deconvolution with the PSF shown in the left image. This profile is the result obtained following 60 iterations of R-L deconvolution. (d) Full image after deconvolution of each line along the fast-scan axis.

Figure 5 conveys the result of the deconvolution we employed to resolve streaked images. Because the distortion from luminescence lifetime is entirely one dimensional, we can separate the image into single pixel slices along the fast-scan axis. Figure 5(b) plots the profile of one such section, from which it can be observed that nanoparticle emission matches well with the fit obtained through single photon counting in a suspension of lipid coated UNPs. R-L deconvolution of these profiles typically converged after 120 iterations, but between 20 and 60 iterations were sufficient to fully resolve UNPs at a lateral resolution roughly equivalent to that of the slow-scan axis (Figs. 6 and 7). The profile plotted in Fig. 5(c) is the result of 60 iterations of deconvolution. The final, reassembled image is shown in Fig. 5(d).

Fig. 6

Deconvolution of luminescence from a single, lipid coated nanoparticle over 120 iterations. Excitation power density for these images was $\sim 850\mathrm{W}/{\mathrm{cm}}^2$. Iteration number is shown in the top right of each image. Images were normalized to the same maximum value for comparison purposes.

Fig. 7

Single fast-scan line taken from images in Fig. 6 and presented in profile. Intensities are normalized to aid comparison. Excitation power density was $\sim 850\mathrm{W}/{\mathrm{cm}}^2$. Iteration number is shown in the top right of each plot.

4.

Discussion

Immediately following coating and conjugation, very little aggregation of the UNPs was observed, as indicated by sizing. Over the course of weeks, some settling occurred in lipid coated UNP samples. For particles coated with 50% PEG-lipid instead of the 5% used here, suspensions have remained stable for over one year. As can be seen in Fig. 5, some aggregation of particles did occur during staining of tissue sections. This manifests as differences in intensity between each discrete cluster of particles. A smaller concentration of PEG-lipid in the UNP coating was chosen to avoid interference with conjugation to anti-EGFR and binding to EGFR in the tissue sample; however, a higher concentration may be preferable to prevent this aggregation.

It is interesting to note that while we observe a significant decrease in the rise component of red luminescence, the decay is relatively unchanged by lipid coating and dispersion in an aqueous medium. This discrepancy is likely the result of a match between short-lived excited states in ${\mathrm{Er}}^{3+}$ and high-energy vibrational modes in either the solvent or lipid coating, resulting in preferential quenching of those states.²⁶ While the contribution of individual time constants to each decay component is complex, increased quenching of the ${}^4{\mathrm{S}}_{3/2}$ or ${}^4{\mathrm{F}}_{9/2}$ excited states may explain the observed effect.²⁷

The method of image acquisition presented here compares very favorably to the methods recently proposed by the Li and van Veggel groups.^16,17 In the former, a confocal pinhole was added to the image acquisition path in order to block out of focus light.¹⁶ This has the advantage of improving axial resolution at high excitation powers and is successful in removing pixel bleeding caused by UNP emission. However, this method is not without disadvantages. UNP emission typically does not reach a maximum until many microseconds after excitation. If the pinhole is large, peak emission is observed at a location down-scan from the actual nanoparticle location and may include emission from adjacent nanoparticles. If the pinhole is small enough to properly locate the particle, peak emission is ignored altogether and much higher excitation power densities are required in order to detect nanoparticle emission. In addition, removing the tail end of emission also removes UNP sensitivity to dynamic events that interact with the long lived electronic states.

In the latter, a different approach to avoiding UNP emission streaking present in scanning microscopy images is demonstrated.¹⁷ This approach instead uses wide-field imaging, and is able to associate the entire rise and decay of UNP emission with a specific location that corresponds well with the particle location. There are two immediately apparent drawbacks to this method of imaging when compared to two-photon scanning microscopy. First, scattering of emission light will result in a slight loss of lateral resolution, increasing with depth into a tissue sample. More importantly, wide field illumination prevents the use of this imaging technique as an optical sectioning technique.

Another method for avoiding the streaking effects caused by long lanthanide luminescence was presented by Faris’s group.¹⁸ This method replaces the PMT detection typically used for multiphoton microscopy with a CCD camera while retaining scanned excitation. This wide-field acquisition results in images that are free of streaking along the fast-scan axis and have the same potential for optical sectioning as a multiphoton microscopy system. When working in thick tissue samples, wide-field acquisition may, however, result in a loss of lateral resolution due to collection of scattered luminescence photons.

The solution proposed here retains optical sectioning and detects the entire rise and decay of UNP luminescence, maximizing the amount of light collected. R-L deconvolution of images like that shown in Fig. 5 was implemented using a fairly simple script in MATLAB. Sixty iterations of this script completed in $\sim 7\mathrm{s}$ using an Intel Core2 Duo E7200 CPU at 2.53 GHz. On the other hand, differences in individual nanoparticle lifetimes will result in small differences in the effectiveness of a set number of R-L deconvolution iterations. Because the deconvolution takes Poisson noise into account, it is also possible for signal from a single particle to be mistaken as part of the streaking from a large aggregate of particles located immediately up-scan. These issues can be in part addressed by implementing a goodness-of-fit test to control the iterative process of deconvolution. Further optimization of the R-L deconvolution and a better processor would likely allow for implementation of this method as an unobtrusive part of image acquisition rather than a post-processing step. It may also be possible to parallelize deconvolution and image acquisition to some extent, such that deconvolution begins on each scan line as soon as it is collected, before image acquisition has finished. The use of a software based solution to the problem of UNP streaking is particularly enticing because it allows commercially available two-photon microscopy systems to be used for UNP imaging without hardware modification.

5.

Conclusion

We have shown that deconvolution by a point spread function determined by the time resolved luminescence of upconverting nanoparticles can greatly diminish streaking caused by their long lived excited states. R-L deconvolution provides a convenient route to retaining fast image acquisition times and optical sectioning. We have also demonstrated here what we believe to be the first use of lipid coating as a viable method for rendering UNPs water dispersible. In addition, this method of coating provides a convenient platform for conjugation to biologically relevant ligands, as we have demonstrated for anti-EGFR. Further improvements in the quality of these particles, such as the use of a core/shell system,¹⁴ will likely result in better image quality at low excitation power densities. It should also be noted that improvement could be achieved by choosing a different excitation source for two-photon upconversion imaging. While two-photon autofluorescence, associated with femtosecond laser pulses, was minimal at 665 nm, the use of a CW or longer pulse width laser would result in effectively no autofluorescence signal and enable acquisition at all ${\mathrm{Er}}^{3+}$ emission peaks. The detected luminescence intensity could also be increased by using ${\mathrm{Tm}}^{3+}$ instead of ${\mathrm{Er}}^{3+}$, resulting in a single, large emission peak at $\sim 800\mathrm{nm}$ instead of two peaks at 546 and 665 nm. This anti-Stokes shifted NIR emission is also preferred for deep tissue imaging due to its low attenuation in biological samples. This report has focused on the development of a technique to correct the lateral resolution of a two-dimensional (2-D) image of UNPs while maintaining the axial resolution enabled by these contrast agents. Future work will apply lateral deconvolution to resolve details in three-dimensional (3-D) deep tissue imaging with UNPs.