1.

Introduction

At the core of many modern industrial processes are flowing suspensions of microparticles, which have to be handled. Important examples are suspended growing crystallites or microorganisms in agitated reactors as well as oil dispersions with metal particles or water droplets in machine systems. Monitoring of these microparticles is indispensable for an in-depth understanding and for optimized operation. Here, light microscopy is used as a gold standard. Conventionally, microscopes are applied ex situ on extracted and prepared suspension samples in counting chambers.¹ In this way, light microscopy can only be carried out at a limited number of time points, and each time on a limited number of static objects, e.g., prepared cells. Both limitations tend to increase the measurement uncertainties with respect to the particle concentration and morphological features.

In contrast to static microscopy, a very large number of cells can indeed be observed by flow cytometry.² This technique generates diffraction and fluorescence signals from individual cells in a prepared suspension jet. Recently, light microscopy has been combined with flow cytometry such that this technique can generate high-resolution cell micrographs with large statistics.³ However, imaging flow cytometry is again based on extracted and prepared samples. It can, therefore, only improve the measurement statistics within a given extracted sample, but it cannot reduce the statistical uncertainties caused by sampling and preparation.

Light microscopy can avoid both shortcomings—uncertainties from sampling and low statistics—provided it is carried out in situ and contact-free. This means that the microscope has to be directly coupled to the suspension via a transparent window without disturbing the suspension’s flow. Sampling and preparation are then replaced by a continuous stream of image data directly from within the suspension. Since new particles are continuously transported through the open observation zone at a rate proportional to the flow speed and to the particle concentration of the suspension, a large number of particle images are accumulated. In this way, the contact-free in situ microscope (ISM) provides large statistics resembling an imaging flow cytometer in this respect. Experimentally, this microscope has indeed been applied for in situ imaging cytometry.⁴

However, these virtues are not sufficient from the user point of view since they require a certain operational complexity. During the operation of a high-resolution ISM, it is necessary to ensure precise optical adjustment on a micrometer scale at all times. This feature is inherited from laboratory microscopes, and it seriously impedes the application of in situ microscopy in the field of routine sensing. Bioreactor probes, e.g., should work reliably even if only a little effort for maintenance and attendance can be provided. Contrary to this, present ISM instruments require regular attendance to adjust the precise distance between objective and optical window in a reproducible way. This effort is needed since the optical window works like the cover slip in a conventional microscope and accordingly must be precisely positioned relative to the objective. Though personal attendance can be replaced by autofocus techniques,⁵ this solution is at the cost of simplicity and causes disadvantageous constraints on the sensor construction.

Here, we present a solution which omits all optomechanical adjustments. This is done on the basis of a specially designed immersion objective whose front surface is directly immersed in the suspension. Consequently, the objective itself constitutes the optical window toward the suspension, eliminating all window-related adjustments. Only the distance between objective and camera sensor (image plane) can be varied. However, due to the geometrical imaging with a magnification factor of $40\times$, the millimeter precision at the image plane implies a micrometer positioning of the conjugated object plane within the suspension. By applying this adjustment-free system to suspended yeast cells (approximately size $5\mu \mathrm{m}$) and animal cells (approximately size $15\mu \mathrm{m}$), we demonstrate submicrometer resolution and significant statistics with respect to the concentration and morphology features.

2.

Optical Requirements

As discussed above, we describe an adjustment-free ISM setup based on a water immersion objective which is corrected for applications without a cover slip. The main optical requirements of such an objective are as follows.

To obtain good statistics, a reasonably large field of view of about 0.1 to 0.2 mm is required. In addition, a numerical aperture $(\mathrm{NA})=0.75$ is needed, since according to Wiedemann et al.,⁴ this value is adequate for establishing submicrometer resolution within the ISM images. Moreover, the scattering and absorption within cell suspensions require the object plane to be very close to the front surface. In this way, we minimize the loss of information due to the scattering of imaging rays by unfocussed cells. The distance between the object plane and the objective’s front is chosen to be $15\mu \mathrm{m}$ so that individual cells can fully enter the object plane.

Standard off-the-shelf immersion microscope objectives are designed for operation within a fluidic environment; however, they are still not ideal for our purpose. First, they are designed for operation in combination with a cover glass and/or with working distances of more than $100\mu \mathrm{m}$, which does not meet the requirements mentioned above. Second, even though they are designed for immersion, they are usually not fully sealed for usage under pressure. Moreover, high-NA immersion microscopes are quite complex since they are typically fully color corrected and field flattened. Another option could be the use of off-the-shelf endoscope objectives, which are sealed and even autoclavable, but they have less optical resolution and are also quite complex and costly.⁶

Since no appropriate off-the-shelf solution exists, it is our goal to find a simple system that still possesses the required imaging quality. To do so, it is helpful to analyze which optical requirements can be relaxed as compared to a standard microscope application.

The transmission characteristic of water suggests an inspection within the visible range and the preferred light source is a luminescence diode. Thus, achromatic correction is only required for a very narrow spectral range of about 10 to 20 nm. Moreover, the object plane for our application does not necessarily have to be flat since the high NA will self-define a small volume within the suspension which is sharply imaged to the flat sensor according to the narrow depth of field. Thus, a slightly curved object plane is acceptable.

In summary, the optical specification of the microscope objective is given in Table 1.

Table 1

Optical requirements of the in situ microscope (ISM).

3.

Optical Design

The narrow spectral range and the allowance of a curved object plane should enable a much simpler design as compared to a standard achromatic and flat field immersion microscope. In fact, a first attempt could be just to employ a single aspheric lens in combination with a planar protection window for our application. It, however, turns out that such a design will be limited to an NA of about 0.5 for the specified field of view. Therefore, the system architecture needs to be slightly more complex to satisfy our needs, especially to provide the desired NA.

Since our requirement on the working distance is such that the object plane shall be located very close to the first planar surface of the system, the optical arrangement is quite comparable to a solid immersion application in microscopes.⁷ Solid immersion lenses in microscopes are used to boost the resolution of a standard microscope by attaching a strongly curved, spherical aplanatic lens of high index to the object. Such an arrangement can provide an additional gain in NA of a factor $n^2$ corresponding to the index of refraction of the solid immersion lens. Transferring this idea to our design results in an arrangement of one single aspheric lens to provide the basic focusing, plus a solid immersion type lens as a front lens to increase the NA. This design is illustrated in Fig. 1(a).

Fig. 1

Optical designs of the immersion microscope, providing $\mathrm{NA}=0.75$ on a curved object plane: (a) design consisting of custom-designed elements and (b) design made from off-the shelf components. Both designs employ one planar-convex aspheric lens in combination with a solid-immersion type front lens.

Indeed, such an arrangement proves to be very well suited for our purpose. Using a high-index LaSF9 planar-convex spherical front element in combination with one aspheric lens element from SK5, a diffraction-limited performance can be achieved across the full field of view for monochromatic light. Moreover, the shape of the aspheric surface can be a simple conic, and the backside of the lens can be kept planar. Since only positive power elements are used, the Petzval curvature remains uncorrected, resulting in an object plane which is curved toward the front lens. The radius of curvature of the object plane for this design is of the order of $r=1.4\mathrm{mm}$. The main residual aberration on this curved plane is the axial color, which in this design type cannot be fully corrected, but only minimized.

After having specified the optical requirements, we address the goal to minimize the complexity of the design and ultimately to minimize the costs of production. In particular, the objective should be based on a commercially available planar-convex aspheric lens. However, most of the available aspheric shapes are designed for collimated input light and a point focus. Although our requirement for the aspheric shape is slightly different, we still are able to find an off-the-shelf aspheric lens which can replace the aspheric lens in our design, only resulting in a residual amount of spherical aberration. Similarly, the planar-convex spherical front lens can be replaced by an off-the-shelf lens. However, the ratio between the thickness and radius of the solid immersion lens, in other words the aplanatic arrangement of the lens, is quite critical to aberrations,⁸ therefore, replacing the front lens by an off-the-shelf part again results in some amount of spherical aberration. To correct the overall sum of residual spherical aberration, a third element in the design becomes necessary. It turns out that a simple thick planar plate of BK7 located inside the converging beam path behind the aspheric lens introduces just the right amount of spherical aberration to compensate the other elements. In summary, we are able to design a high-NA microscope system by the arrangement of off-the-shelf components as listed in Table 2.

Table 2

Design prescription and off-the-shelf-part list of the ISM.

The design layout is shown in Fig. 1(b). The corresponding MTF characteristic in Fig. 2 proves that with this design, a performance close to the diffraction limit can be achieved.

Fig. 2

Performance of the design shown in Fig. 1(b). The MTF characteristic on the left shows that the design is very close to being diffraction limited. The transverse aberrations on the right reveal that the main aberrations result from axial chromatic aberrations (here secondary wavelengths of $\pm 10\mathrm{nm}$ from the central wavelength 650 nm are shown).

From the transverse aberrations depicted on the right of Fig. 2, it becomes apparent that axial chromatic color is the limiting aberration in this design. For our application, i.e., a spectral bandwidth of $\pm 10\mathrm{nm}$, this amount of color aberration is acceptable; however, for broadband light sources, this might be a severe limitation. One way of reducing the axial color in the design can be to replace the single lenses by an achromatic doublet; however, this again increases the complexity of the design. Another option, therefore, is to use the planar backside of the asphere for the placement of a diffractive optical structure. Since the dispersion of such a diffractive element is opposite to the material dispersion, a compensation of axial color seems possible.⁹ Moreover, the diffractive element could in addition provide the required aspheric wavefront correction, such that the optical design for the polychromatic application could potentially consist of only two spherical lenses plus an imprinted diffractive structure.

4.

Fabrication and Experimental Setup

By replacing all lenses in the above design by off-the-shelf components, we have avoided manufacturing individual lenses; however, we still need to check whether the available element tolerances are sufficient and how to mount the system according to our needs. The effect of the element tolerances, as listed in Table 2, as well as centering tolerances of $\pm 0.01\mathrm{mm}$ have been considered in a statistical tolerance analysis. The main result is depicted in Fig. 3, where the expected drop in the MTF performance (2 sigma probability, i.e., 95% of the samples will be better) for the worst field point for a spatial frequency of $250\mathrm{lp}/\mathrm{mm}$ in the object space is illustrated.

Fig. 3

Influence of the element tolerances (focal length and thickness tolerance) and centering tolerances on the MTF performance at a spatial frequency of $250\mathrm{lp}/\mathrm{mm}$ in the object space.

Here, we have omitted the influence of the plane parallel window since the tolerances of this element are uncritical. For the other two off-the-shelf components, i.e., the asphere (Asph) and the plano-convex (PCX) lens, the analysis shows that the influence of the asphere tolerances is by a factor of 10 less critical than for the PCX lens. Moreover, if no additional compensator (other than refocusing) is used, the effects of the PCX tolerances are unacceptable since the thickness tolerance alone would result in an MTF drop of up to 40%. However, if the air gap between the asphere and the PCX lens is used as a compensator, the effect of the tolerances can be limited to an acceptable MTF drop of around 10%. This includes centering tolerances of 0.01 to 0.02 mm, which are tight but can be achieved by standard manufacturing procedures. Further tolerance simulations at other spatial frequencies result in a similar MTF drop, such that the as-built system performance is expected to be in the MTF range as illustrated by the dashed line in Fig. 2.

An additional requirement is to provide a flat and tight interface toward the bioreactor. To achieve this, the front lens is glued into the mount. All other components can then be filled and mounted from the backside by standard retainer rings as illustrated in Fig. 4. This mounting technique allows us to use the air gap between the asphere and the front lens as a compensator for symmetric tolerances as discussed above. Thus, tolerances in the element thicknesses or radii (especially of the PCX lens) can be compensated by shimming the corresponding distance ring.

Fig. 4

(a) Illustration of the mechanical mounting of the objective, (b) setup of a submicron-resolution microscope using the new objective. All elements are roughly positioned by table tripods and no precision adjustment is necessary.

In addition to the effects of production tolerances, thermal effects on the optical performance need to be analyzed for applications at elevated temperatures. It turns out that up to 100°C, the main induced error is a small focus shift, which is acceptable in our setup since the object plane is not fixed. However, to avoid mechanical stress on the optical parts, the mounting material was chosen to be titanium.

To demonstrate the adjustment-free operation of this new system, we use a straightforward experimental setup that requires no precision as shown in Fig. 4. A plastic cuvette with a magnetic stirrer serves as the container of the moving suspension. The above described objective is directly immersed in the suspension by attaching its front flange to the border of a circular opening in the cuvette’s wall with silicon glue. Opposite and concentrically to the objective, flash illumination emerges from the flat end of a 1-mm Polymere-optical-fiber. The pulses are emitted from a pigtailed light-emitting diode (LED, $650\mu \mathrm{m}$, pulse width $2\mu$, DieMount GmbH). The light pulses⁴ are synchronized with the shutter time of a CCD camera (BASLER 102A).

The camera is positioned at a distance of 120 mm (tolerance 1 mm) using a table tripod. The orthogonality of the camera sensor versus the optical axis is visually established with a tolerance of approximately 5 deg. An open tube in front of the camera serves to shield off most of the environmental light. Since we use a synchronized shutter time of approximately 0.1 ms, 100%—shielding from environmental light is not necessary.

This simple arrangement shows how uncritical the mounting of this microscope is. It implies that all components can be rigidly assembled and attached to the wall of a reactor with no critical positioning or adjustment issues.

5.

Application Results

5.1.

Microscopic Resolution

To demonstrate and quantify the objective’s resolution, the agitated cuvette of Fig. 4 is filled by a suspension of Saccaromyces cerevisiae cells (baker’s yeast) with a concentration of approximately ${10}^7\text{cells}/\mathrm{ml}$. Examples of moving yeast cells imaged by the ISM are shown in Fig. 5. The diameter of these cells is approximately $5\mu \mathrm{m}$. Subcellular structures are imaged with submicrometer resolution. Figure 5(b) zooms into the details of one cell so that individual pixels can be seen. Each pixel covers a square of $0.16\mu \mathrm{m}\times 0.16\mu \mathrm{m}$ in the object field. Image analysis reveals that within the image, structures of $0.3\mu \mathrm{m}$ size (i.e., spatial frequencies of $1600\mathrm{lp}/\mathrm{mm}$) are visible with a contrast of better than 10%. Comparing this to the expected MTF performance at this frequency (Fig. 2), we can conclude that the optical system performance is indeed in between the ideal performance and the expected as-built performance.

Fig. 5

Saccaromyces cerevisiae cells imaged by the in situ microscope (ISM) in flowing suspension, using the setup according to Fig. 4: (a) 22 micron image section; (b) zoomed section of the central cell corresponding to 6 micron diameter.

5.2.

In Situ Morphometric Applications

5.2.1.

Morphometry at submicrometer level

A basic morphological feature of an imaged microobject is its area projected onto the image plane. If that area is interpreted as a circle, its equivalent diameter is obtained as a random variable over the ensemble of all cell images and it serves as a feature value describing the cell size. The projected area can be extracted from sharp cell images as described in Camisard et al.¹⁰ Mean values ED over a certain number of cells can then be interpreted as a size measurement with a statistic uncertainty depending on the sample size.

For this test, we capture images from a suspension of Saccaromyces cerevisiae in the agitated cuvette as a function of time. ED values are extracted from samples of 400 cells each and registered as a function of time. The resulting online curves constitute monitoring of the average object size.

Figure 6 displays a size measurement taken over a time span of 1 h. There is no exterior influence on the suspension except that at the time point of 27 min, an osmotic pressure step of approximately 20 MPa is caused by the addition of NaCl. In accordance with previous ISM experiments,¹⁰ we observe a step decrease of the cell’s ED by approximately $1\mu \mathrm{m}$. In this curve, the amplitude of statistical noise of the ED value amounts to approximately $0.2\mu \mathrm{m}$. Hence, morphological changes of the ensemble average ED are monitored with a resolution of at least $0.2\mu \mathrm{m}$. In theory, this resolution for measuring morphological changes can further be improved by more statistics from faster image generation and processing. It is only the exchange time of the suspension in the virtual probe volume that puts an upper limit with respect to meaningful frequencies of image acquisition.

Fig. 6

Mean cell size in a suspension of Saccaromyces cerevisiae cells as function of time. Each data point represents the mean equivalent diameter in a sample of 400 cell images captured by the ISM in the agitated cuvette (see Fig. 4). At the time point of 27 min, the cells shrink due to an osmotic pressure step of approximately 20 MPa, caused by the addition of NaCl.

In summary, the adjustment-free ISM allows submicrometer morphometry in real time.

5.2.2.

Cell concentration

The first quantitative application of a contact-free ISM sensor was a concentration measurement in yeast suspension.¹¹ To test this method using the adjustement-free ISM version, we prepared cell concentrations by pipetting a dilution series from a known original concentration of Saccharomyces cerevisiae in water. Typical ISM-images of focused yeast cells are shown in Fig. 5. For this test, we determine the number of sharply imaged cells per image (NpI), averaged over 800 images. The counting of sharply imaged cells in each image is carried out by using the image evaluation algorithm which has been described in detail in Wiedemann et al.⁴ It eliminates blurred cell images and counts the left over sharp cell images. In this way, the algorithm implicitly establishes a virtual probe volume by depth of focus.^4,11

The original (i.e., the highest) concentration was determined using a hemocytometer within an uncertainty of 7% which is typical for the use of hemocytometers. The quotients of diluted concentration over original concentration are expected to be uncertain by approximately 5% mainly due to reading uncertainties during the pipetting.

Figure 7 displays the expected linear correspondence between the NpI data and the prepared concentrations. To within the experimental uncertainty, the fitted line agrees well with the hypothesis that the sample volume is constant so that the NpI value is proportional to the actual concentration. By definition, the gradient of this line is equal to the virtual sample volume based on automatic elimination of blurred cell images. Here, we obtain $(0.18\pm 0.04){10}^{-3}{\mathrm{mm}}^3$. From this value, the depth width of the virtual probe volume can be calculated as $(9.4\pm 0.5)\mu \mathrm{m}$. (The calculation is based on a 2/3″ CC sensor and an optical magnification of 40.)

Fig. 7

Correlation of counted cell number per image versus prepared cell concentration. The horizontal error bar at maximum concentration indicates the SD of the absolute concentration as measured five times by applying the Neubauer chamber to the same suspension at approximately $1.9\times {10}^7\text{cells}/\mathrm{mL}$. For diluted concentrations, no horizontal error bars are shown. It is only the deviations of the relative values (relative to the maximum value) that matter here. These are generated by the dilution procedure and expected to be below 5% of the nominal values. The vertical error bars represent the SD of the measured NpI values if they are repeatedly computed, where each NpI value is obtained from 800 new images from the same suspension sample.

In summary, the new objective allows concentration measurement in suspensions by microscopic counting just as with previous ISMs,^4,11 but without any effort for adjustment.

5.2.3.

Animal cells, viable, or nonviable

Animal cells are relatively large (e.g., CHO cells, typically $15\mu \mathrm{m}$), and their ISM images exhibit characteristic morphological features in the submicrometer range.

Figure 8 shows typical cell portraits taken with the adjustment-free ISM from viable suspended CHO cells (cell line DP12) in the agitated cuvette shown in Fig. 4. With respect to optical resolution, this imaging of animal cells compares well with imaging based on precisely adjusted laboratory microscopes or ISMs.^4,12 Using Hybridoma and Jurkat cell lines, Wiedemann et al.⁴ have shown that viable and nonviable animal cells can be discriminated by the information entropy inside cell borders in images such as those here.

Fig. 8

CHO cells (cell line DP12) in flowing suspension observed by the adjustment-free setup shown in Fig. 4. The suspension is taken from a 2-day-old culture with 97% living cells according to reference measurements. Therefore, these cells show the typical morphology of viable CHO cells.

6.

Conclusions

By analyzing the main optical requirements, we were able to design and build a simple and low-cost objective for suspension immersion allowing adjustment-free in situ microscopy. We proved that although we employ only off-the-shelf optical components, submicrometer resolution is obtained in images of individual microobjects. The quantitative image evaluation produces large statistics of microobjects within minutes. On this basis, the ensemble average of the size of microorganisms is monitored with a resolution of $0.2\mu \mathrm{m}$.

Consequently, both visual in-depth information and morphometry of microobjects in real time are combined in an attendance-free rigid microscopic sensor device.

Notably, the new objective enables for the first time a microscope sensor for cell concentration (biomass) and imaging cytometry in bioreactors without attendance with respect to adjustment. Experimental work is in progress to apply this technique for in situ probing of the viability (percentage of living cells) of animal cells in suspension cultures and for monitoring the filamentous bacteria in wastewater facilities.

Encouraged by these results, our next step is to further improve resolution and image quality. Presently, the main cause of image blur is due to the spectrum of the LED illumination, which is not monochromatic but has a width of approximately 10 nm. By including a diffractive optical element for wavelength correction, the objective’s design will be subject to further development to increase its resolution, while still keeping its simplicity.