2.1.

Observation of AOM

Adding an annular ultrasound transducer to the optical path enables AOM of the fluorescence signal collected by a conventional TPM system [Fig. 1(a)]. The US focus (characterized in Fig. S1 in the Supplemental Material) is co-registered with the optical scan volume, and both foci are co-translated when focusing to different planes by coupling the transducer to the objective (see Appendix). The TPM field of view is limited to within the US focus, and raster scanning is used to acquire the spatial fluorescence intensity images with pixel-rates that are $\sim 20$ times higher than the US period [Fig. 1(b)]. We imaged clusters of fluorescent microspheres while transmitting a continuous wave (CW) US beam and observed a large fluorescence modulation [Fig. 2(a), top left, and Fig. S2(a) in the Supplemental Material]. In single line scans from the image, fluorescence oscillates below its unmodulated level by up to 50% during US transmission. This AOM leads to visible high-contrast “fringes” in the raw raster-scanned images [Fig. 2(a)].

Fig. 1

UMAMI system. (a) System schematic: a $\sim 2.1\mathrm{MHz}$ annular US transducer is attached to the objective’s front aperture of a conventional TPM system (left) via a 3D printed holder (right) that co-registers the optical and US focal planes. (b) Expanded view of the relative size of the US (purple/gray) and optical (red) foci. Illustration of the pressure wave modulation during a line scan (right), highlighting how temporal variation in US pressure results in spatially patterned AOM.

Fig. 2

Giant AOM effect and its characteristics. (a) AOM in an agar-embedded bead cluster (top, left). The 2.1 MHz US wave is slowly gated on (red) and off (green) within the frame to highlight the modulation effect. The modulation of fluorescence from a representative line (red) as compared to the unmodulated baseline (green) demonstrated large fractional modulation (top, right). An in vivo AOM image and the modulation of fluorescence of a single line are shown as well (bottom). (b) Signal frequencies 0 to 5 MHz from an AOM in vivo frame. While the US is turned off, the image frequencies are predominantly contained around the DC; however, with the US on, large peaks at the US fundamental frequency and second harmonic are observed. (c), (d) The modulation of the fluorescence signal at the US fundamental frequency of 2.1 MHz is proportional to the US pressure’s amplitude (top, left) and independent of laser excitation intensity (top, right). (Seven beads, 24 frames averaged each.) (e), (f) Similar results obtained in vivo from (eGFP) labeled neurons $200\mu \mathrm{m}$ from the brain surface. The blue dashed lines represent linear fits, while the green dots show respective baseline (US off) values.

We next characterized the dependence of this effect on the US pressure and the laser intensity. Modulation ratios at the US frequency were generally independent of changes in the fluorescence signal driven by the laser intensity but increased roughly linearly with acoustic pressure [Figs. 2(c) and 2(d)], similarly to previously reported work on single photon fluorescence AOM;^14,16 in contrast, observed second harmonic AOM signals scaled quadratically with pressure (Fig. S3 in the Supplemental Material). In an important first achievement for AOM fluorescence studies, we observe this strong modulation in vivo when imaging enhanced green fluorescent protein (eGFP) expressing neurons with low US pressures ($<500\mathrm{kPa}$), with similar dependencies on pressure and laser intensity [Figs. 2(e) and 2(f), bottom].

2.2.

Extracting Modulation Images

We then explored whether the strongly observed AOM effect could be harnessed to provide significant US modulation assisted multiphoton imaging (UMAMI) enhancements in TPM image quality. We developed a simple post-processing procedure to extract the modulated component of the images by filtering raster-scanned lines in the frequency domain [Fig. 3(a); see the Appendix for details of this digital homodyne detection].^9,17 This demodulated signal resulted in significant contrast-enhancement (the ratio of features to background). The enhancements are readily visible in single demodulated frames of eGFP-expressing neurons in vivo [Fig. 3(a), bottom left] and the increases in SNR with lower frame rates and higher pressures are seen [Fig. 3(a), bottom right, and Fig. S4 in the Supplemental Material]. In beads, the demodulation process reveals that the modulated signal is highest near edges showing how UMAMI enhances edge features [Fig. 3(b)].

Fig. 3

Demodulation for UMAMI imaging. (a) Computational strategy for image demodulation. A series of modulated images are motion corrected, transformed to the Fourier domain, and band-pass filtered around the fundamental US frequency (see Appendix). Band-passed signals are shifted to the low-frequency spectral range and inversely transformed back to the image domain. In step 2, demodulated frames are then averaged as needed until sufficient SNR is reached (right), providing additional information to the baseline (unfiltered) images. (b) Example demodulation process of a fluorescent bead embedded in agar and a non-fluorescent bead embedded in fluorescein-laden agar (left and right, respectively) (24 frames averaged). Original images are unfiltered and have no AOM (top). Demodulated images from when the US is on show pronounced signals at the edges of the beads (middle). When the US is turned off and images are demodulated, no signal is observed (bottom).

2.3.

Model

In a conventional TPM line scan, the fluorescent profile of an object is illuminated by an (approximately) constant velocity linear scan. (Although a true resonant scanner line scan is not linear, the imaging software does not acquire data at the edges of the scan allowing for a more linear approximation of the scan speed to be used). With UMAMI, we reasoned that a putative mode of interaction leading to robust AOM of the fluorescence is based on sinusoidal displacements of the position of the fluorophores relative to the optical focus at the US frequency during the scan [Fig. 4(a)]. The effect of this point spread function (PSF)-fluorophore displacement is that fluorophores near intensity boundaries will be more strongly modulated than fluorophores in “fluorescently flat” areas as they rapidly move in and out of focus [Fig. 4(b)]. To demonstrate where AOM modulations due to tissue displacement are expected to occur, we performed numerical simulations of a $10\mu \mathrm{m}$ bead scanned with a UMAMI-like scheme, observing the appearance of fringe patterns similar to our experiments [Fig. 4(c)]. Bandpass filtering of the modulated signal around the US frequency shows how the modulation amplitude varies across the field of view or a bead-like intensity profile. Using our described demodulation procedure, we extract a demodulated signal profile from these simulations revealing enhanced edge features. We can also recover to a good approximation of the original bead signal profile by performing bandstop filtering (1.8 to 2.4 MHz) on the UMAMI scanned signal. The demodulated signal profile at the bottom of Fig. 4(c) matches that of an actual bead as in Fig. 3(b), middle. UMAMI results in a new signal profile distribution of the bead, not that of the physical bead or fluorophore distribution but rather highlights object edges. In addition, the fast-sampling rate required to detect the modulations is only along the axis of the fast scan, in this case in the $x$-direction. If the fast and slow axes were swapped the enhancements would be in the other direction. Though other AOM mechanisms are not ruled out here, UMAMI acts on fluorescent derivatives. During UMAMI there are two parts of the scan, the original raster scan and the AOM effects on the raster scan. Mathematically, this can be derived by examining the effect of adding a sinusoidal displacement $b\sin (\omega t)$ between the PSF and fluorophore to a raster scan of the object’s fluorescent profile, which can be described as a function of space $x$, itself a function of time, $t$, and the scan velocity, $c$. The small sinusoidal deviations can be approximated with a Taylor series expansion:

Fig. 4

UMAMI mechanism – conceptual model and simulation. (a) Comparison of a scan’s position versus time for a normal raster scan and a sinusoidally displaced UMAMI scan. (b) Schematic depiction of the origin of UMAMI generated fluorescent signals/fringe pattern. During a single line scan, fluorescent signals near the bead’s edge are sinusoidally modulated due to the movement across the PSF’s boundaries (points B and C) while center loci remain relatively constant during spatial deflection and have no resulting signal modulations (point A). (c) Top, the profile of a simulated bead under a regular raster scan versus UMAMI scan. Middle, the band-passed UMAMI scan shows that the modulation amplitude is largest at the edges, where the fluorescent derivatives are greatest, and minimal at the center of the bead. Inset: spectral content of the band-passed modulation signal. Bottom, the results of applying our demodulation procedure and a bandstop filter on the simulated UMAMI scanned signal showing enhanced edges as well as recovery of the original profile. (d) Theoretical dependence of the modulation ratio on the displacement amplitude. The displacement amplitude is putatively proportional to US pressure, leading to an overall linear dependence on pressure.

Here, a Taylor series expansion of the small sinusoidal deviations shows that the sinusoidally modulated signal component scales with the spatial derivative of $f$ and the sinusoidal modulation’s amplitude. Acoustically induced particle displacement $\delta$ is related to the acoustic pressure $p$, by $\delta =p/\rho \omega c_0$, where $\rho$ is the medium density and $c_0$ is the speed of sound in the medium. Since particle displacement is proportional to pressure, the fluorescent derivative in the expansion will be modulated at the US frequency and scale proportionally with pressure/displacement [Fig. 4(d)]. However, this is only a simplified model of three-dimensional (3D) vibrations as a one-dimensional (1D) vibration, so it also neglects other interactions (acoustic, thermal, etc.). This is likely the reason why the simulated signal oscillates around the baseline fluorescence while the experimental signal seems to modulate beneath the original signal. Nevertheless, this simplified model reproduces some of the main features that were observed in this AOM phenomena (Fig. 2).

2.4.

In Vivo Image Enhancements

We next generated 3D image stacks of eGFP- and GCaMP6s-expressing neurons in densely labeled brain tissue. Demodulated images show increased out-of-focus suppression and contrast [Fig. 5(a), left]. Of note, demodulated images introduce marked improvements in detail (the ability to discern structural features in the image). The observed pattern appears to match the underlying nuclear excluded fluorescence; because the fluorescent protein is only cytosolically expressed, background contamination often appears as intranuclear fluorescence. This contrast-enhancement effect appears to reduce such contamination and, therefore, may be beneficial in functional ${\mathrm{Ca}}^{2+}$ imaging where signal mixing due to background is a critical limitation (however, note that it is unclear whether some of this pattern enhancement is solely due to out-of-focus suppression or also results from fluorescence gradient). The enhanced contrast in our images was quantified by comparing line-profiles of neuronal features between unmodulated and demodulated images [Fig. 5(b)]. These profiles clearly demonstrate enhancements in image contrast at neuronal borders and reveal internal features, such as nuclear exclusion. They also demonstrate that UMAMI can generate high-contrast image features previously obscured by uniform background signals, without affecting the imaging frame rate. In addition, in GCaMP6s neurons, we found a strong correlation between the demodulated signals and the baseline neuron GCaMP signals (correlation coefficient, $r=0.96$; see Fig. S5 in the Supplemental Material). This linear correlation indicates that the modulation depends on the concentration of excited fluorophores¹⁴ and could potentially be used in the context of functional imaging.

Fig. 5

UMAMI of cortical neurons in vivo reveals enhanced detail and contrast. (a) Demonstration of improved optical sectioning in densely labeled (GCaMP) cortical layer 2/3 neurons (left). $Z$-stacks for baseline and during AOM. Right: the higher zoom stack highlights rejection of out-of-focus fluorescence, revealing the nuclear exclusion of eGFP. (b) Baseline and demodulated neuron images: green (baseline) and red (AOM demodulated) lines show enhanced contrast, as well as background rejection, revealing nuclear fluorescent protein exclusion.

4.1.

Principle of UMAMI

Using ring-shaped transducers, the pressure waves at MHz modulation frequencies can be focused into sub-mm foci in murine brains while allowing for co-axial two-photon imaging within the same volume. In mice with a cranial window, we observed that insonation at $\sim 2\mathrm{MHz}$ causes a large AOM of two-photon generated fluorescence, at imaging depths of up to $300\mu \mathrm{m}$. When the acoustic pressure wave propagates through the tissue as the microscope focus is raster-scanned, this temporal modulation becomes spatially resolved. As the scan sampling rate is much higher than the transducer’s resonant frequency, a sinusoidal modulation of fluorescence is measured. The modulation spatial period is determined by the ratio of the lateral scan velocity to the US frequency. Either parameter can be tuned to increase or decrease the spatial frequency of the modulation.

4.2.

Two-Photon Microscopy

All images were acquired on a custom two-photon microscope based on the Janelia MIMMS 2.0 design, using a conventional optical path and commercial lenses (Thorlabs). Light from a femtosecond laser (Coherent Chameleon Discovery) was raster scanned through a 4× scan-tube relay system with a resonant scanning galvanometer pair (Cambridge 8 kHz resonant scanner and 6015H 5 mm linear galvanometer). Raster scan patterns were imaged to the sample plane with a long-working distance (8 mm), two-photon optimized objective (Olympus XLPLN10XSVMP, 10×, 0.6 NA). Furthermore, to avoid cross-talk in the spatial frequency domain, we increased the image magnification, ensuring spatial sampling rates high enough to support the fluorescence signal modulation frequency (thus imposing an upper limit on how large the field of view can be). Fluorescence at wavelengths $>500\mathrm{nm}$ was excited at 920 nm for all experiments and collected by a PMT (Hamamatsu H11706P-40). Signals were amplified (Femto DHPCA-100) and digitized with a high-speed DAQ (National Instruments PXIe-1073) at 40 MHz through ScanImage 2019 (Vidrio Technologies). For imaging-phantom studies, fluorescent microspheres (Polysciences Fluoresbrite YG) of various sizes ranging from 0.1 to 10 mm were embedded in 3% agarose. Mice with eGFP or GcaMP6s were imaged while they were awake and head-fixed in in vivo experiments. We disabled the adaptive pixel binning default setting in ScanImage, which corrects for the nonlinear dwell times associated with sinusoidal resonant scanning, in order to ensure accurate time-domain representation of fluorescence signals (Fig. S6 in the Supplemental Material). (Although we did not compensate for this effect in the image domain because we were interested here only in local image effects, the resulting geometric distortion can be simply compensated for by applying a sinusoidal stretch and uniformity correction in the $x$-direction.) All images were taken at $1800\times 512\text{pixels}$.

4.3.

Ultrasound

To ensure colinear and confocal AOM of two-photon fluorescence, we selected a single-element annular PZT-4 transducer (TRS Ceramics, 2.1 MHz height-mode resonance frequency). We selected the geometry (inner diameter 5 mm, outer diameter 10 mm, and thickness 1 mm) of the transducer to enable two-photon imaging through the annulus without interfering with the optic wave, and with an appropriate focal length and frequency to ensure overlap of the acoustic focus with the TPM raster scan. The US focus was mapped in 3D to determine its focal dimensions and location (Fig. S1 in the Supplemental Material). The 3D mapping procedure was done in a water tank with the US transducer attached to a three-axis micromanipulator (MP-285A, Sutter Instruments) and a hydrophone (HNR-0500, ONDA Corporation) to report pressures in the volume below the transducer. The hydrophone came pre-calibrated with an MPa to mV conversion factor, which was used to calculate the pressures from the reported voltages recorded with an oscilloscope (Model 2555, BK Precision Corporation). The hydrophone was then centered within the US PSF, and the driving voltage of the function generator was increased incrementally to obtain a voltage to pressure calibration curve used for experiments [Fig. S1(b) in the Supplemental Material].

A 3D printed housing was then created to fix the transducer to the objective so that the US focus was centered and overlapped with the objective’s focus. The upper portion of the 3D printed housing had an internal diameter slightly larger than the objective’s external diameter so it fit snugly on the front of the objective. The numerical aperture of the US transducer is larger than the optical numerical aperture so as to not obstruct the beam. US gel (Aquasonic, Biomedical Instruments Inc.) was used to couple the objective and US transducer to the cranial window to allow proper optical index matching and acoustic coupling, respectively. A frequency generator (Rigol DG1022) was triggered at the start of image acquisition sequences to generate a 2.1 MHz sinusoidal driving signal to a 30 W RF amplifier (Mini-Circuits LZY-22+). Driving signals were not phase-locked to the raster line rate, leading to phase shifts across lines. US was activated as a CW sequence at a duty cycle of 5% or 6%, generating sequences of 6 modulated frames (200 ms) preceded and followed by 90 or 116 (3.0 or 3.8 s) unmodulated frames. The only exception for this US duty cycle is Fig. 2(a) where the duty cycle was set to 50% to help demonstrate the AOM effect compared to when the US is off. US application was only turned on for short periods of time to prevent fluorescence decreases due to fluoro-thermal effects.¹⁸

4.4.

Demodulation

All image processing and demodulation was done in MATLAB (version 2020a, The MathWorks Inc.). We first took the acquired images and transformed them into the spatial-frequency domain by taking a 1D Fourier transform (with MATLAB’s built-in fast Fourier transform function) along the scan direction. Each line of the image was band-pass filtered at 2.1 ± 0.3 MHz (in the Fourier domain) by setting all frequencies outside of the range to zero and then shifted to be centered around DC. We then computed the demodulated acousto-optic image by using a 1D inverse Fourier transform of this complex signal.^17,19 To remove any residue from the two-photon image and the background noise from the same AOM frame, the frequencies just above and below the band-passed fundamental frequency were likewise band-passed, averaged, and then subtracted from the signal. We performed motion correction on the demodulated images with NoRMCorre.²⁰ The demodulated image was multiplied by a factor of two due to the filtering process only taking the positive end of the frequency spectrum, while the original image includes both positive and negative frequencies. We calculated the modulation ratio defined as the demodulated image divided by its baseline image. Unless noted otherwise, all images are shown scaled to their best contrast.

4.5.

Animal Preparation

All procedures were approved under the New York University Langone Health Institutional Animal Care and Use Committee. Male and female C57BL/6J mice (Stock No. 000664, Jackson Laboratories) between 2 and 6 months old were used in all experiments and handled in accordance with institutional guidelines ($n=2$ mice). During the surgical procedure, mice were anesthetized with 1.5% isoflurane. The scalp was cleared of fur, and then the skin was removed to expose the skull. A 3 mm diameter circular craniotomy was performed over the primary visual cortex using an air-driven dental drill (Dentsply Sirona Midwest Tradition) to expose the brain. Adeno-associated viral vectors (serotype 2/1) encoding for eGFP or GCaMP6s ($0.5\mu \mathrm{L}$, ${10}^{13}$) were injected into exposed brain using a stereotactic syringe pump (Drummond Scientific Company Nanoject 2) at $0.05\mu \text{L}/\min$. After viral injection, a circular glass coverslip 3 mm in diameter was implanted as a cranial window to allow for chronic imaging. The window was secured in place in the skull with cyanoacrylate glue and a self-curing dental cement (Parkell C&B Metabond Quick). Head restraints were created from two metal rails positioned above the lambda and bregma cranial sutures. The two metal posts were placed transversally on opposite sides of the cranial window and secured with the same dental cement. Each animal recovered for at least 1 week before experimentation and was imaged after the third week of virus-driven expression.