2.1.

Plant Models and Biochemical Assays for Chlorophyll Quantitation

A series of 24 leaf samples from A. thaliana, a well-known model plant, were utilized for covering a wide range of total chlorophyll content from 0.17 to $1.68\mu \mathrm{g}/\mathrm{mg}$. A. thaliana ecotype Col-0 was cultivated at a light intensity of $100\mu \mathrm{E}{\mathrm{m}}^{-2}{\mathrm{s}}^{-1}$ at 22 °C under a photoperiod of 16-h light/ 8-h dark. Rosette leaves were detached from 4-week-old plants, were incubated in a solution of 3-mM MES (2-[$N$-morpholino]ethanesulfonic acid) at pH 5.7, and were kept under dark for up to 6 days. For conventional biochemical assays, we extracted chlorophyll with 95% ethyl alcohol after incubating the samples at 70 °C for 1 h. Then, absorbance at 665 and 649 nm was measured using a UV/visible spectrophotometer. Finally, total chlorophyll content (i.e., chlorophyll $a$ and chlorophyll $b$) was calculated in the unit of $\mu \mathrm{g}/\mathrm{mg}$ (i.e., chlorophyll/fresh weight) as previously described.¹¹

2.2.

Laboratory Hyperspectral Imaging System

For obtaining experimental spectra from leaf samples, our recently developed system described elsewhere was used.^12–14 This system allowed us to acquire a matrix of reflectance intensity (also known as a hypercube) $r(x,y,\lambda )$ as a function of the position $(x,y)$ and the wavelength $\lambda$ of light. The specification included a transverse resolution of $\sim 100\mu \mathrm{m}$ with a field of view of $15\times 15{\mathrm{mm}}^2$, an imaging depth of $\sim 1\mathrm{mm}$, and a spectral range of 400 to 770 nm with a spectral resolution of 2 nm. A back-directional (angular) filtering scheme in the detection part collected the light reflected from the sample within a narrow solid angle in the exact backward direction. Importantly, this configuration avoided spectral variations originating from different systems, because reflectance spectra, in particular scattering components, are highly sensitive to illumination and detection geometries.¹⁵ It should be noted that this aspect is crucial to apply a hyperspectral image reconstruction algorithm, trained by hyperspectral data obtained from the laboratory system, to a spectrometerless imaging system.

To remove the stray background light and to compensate for the system responses, the following procedures were implemented:^13,16 First, after the acquisition of a raw intensity matrix from the sample $r_{\text{specimen}}(x,y,\lambda )$, a background intensity matrix $r_{\text{background}}(x,y,\lambda )$ without the sample was measured and then subtracted from the raw intensity matrix. Second, by placing a reflectance reference standard (Labsphere, North Sutton, New Hampshire) on the sample stage, a reference intensity matrix $r_{\text{reference}}(x,y,\lambda )$ was measured. Because the reflectance reference standard had flat spectral and uniform spatial responses in the entire visible range, $r_{\text{reference}}(x,y,\lambda )$ captured the entire system responses. Third, $r_{\text{specimen}}(x,y,\lambda )$ was normalized by $r_{\text{reference}}(x,y,\lambda )$ such that

2.3.

Spectrometerless Imaging System Using a Three-Color CCD

A working prototype of the spectrometerless hyperspectral imaging system was also constructed, as shown in Fig. 1. To directly use a hyperspectral reconstruction algorithm trained using the laboratory system, it was important to minimize any spectral variations resulting from different illumination and detection configurations.¹⁵ Thus, the illumination and detection configurations of the laboratory system were mimicked in the prototype system. In particular, a telecentric lens with coaxial illumination (magnification of $0.3\times$, Schott Moritex, Japan) allowed us to image the light scattered from the sample in the exact backward direction with respect to the incident light, acting as back-directional angular gating in the reflection mode.¹⁷ A white-light LED (${0.315}^{\prime \prime }$ LED Spot Light, Edmund Optics, Barrington, New Jersey) was coupled to the telecentric lens via a fiber-optic light guide and was illuminated onto the plant sample. The light reflected from the sample was collected using the same telecentric lens mounted with a three-color CCD (Color Grasshopper3, Point Grey Research, Richmond, British Columbia). The system had a field of view of $70\mathrm{mm}\times 60\mathrm{mm}$ with a pixel size of $56\mu \mathrm{m}$. Similarly to the hyperspectral measurements, an RGB intensity matrix was normalized by a reference signal from the reflectance reference standard to compensate for the spectral and spatial responses of the entire system, including the light source, the fiber-optic light guide, the telecentric lens, and the CCD camera. Thus, it should also be noted that this resultant intensity matrix was relatively independent of the spectral responses of the light source and the camera.

Fig. 1

(a) Schematic diagram of the spectrometerless hyperspectral imaging system. Inset: Spectral profile of the white-light LED. (b) Photograph of the current working prototype.

2.4.

Reconstruction of Hyperspectral Image Data from RGB Data

The relationship between a full reflectance spectrum and an RGB reflectance spectrum in each $(x,y)$ position can be modeled such that^8–10

Suppose that a reflectance spectrum $r$ and a corresponding RGB spectrum $x$ are collected for each sample. Then, ${\widehat{R}}_{m\times N}$ and $X_{m\times 3}$ can be formed by stacking the spectra from several independent samples, where $m$ is the number of different samples and $N$ is the number of wavelengths. This reconstruction problem is to learn a conversion matrix $T$ from a training set such that

3.1.

Hyperspectral Chlorophyll Imaging

To map out detailed spatial distribution of leaf chlorophyll concentration in the actual unit of $\mu \mathrm{g}/\mathrm{mg}$, we utilized a chlorophyll spectral index and converted the spectral index to an absolute value of chlorophyll concentration. First, for plant chlorophyll content quantification, we exploited the extensively used spectral index (SI) for chlorophyll content²⁶

Fig. 2

Correlation between the spectral index SI for chlorophyll content and the biochemical assays. The solid line is a linear regress fit and the dotted lines are 95% confidence intervals.

Fig. 3

Nondestructive, quantitative, and label-free chlorophyll imaging of individual leaves. (a) and (c) Photographs of A. thaliana samples. (b) and (d) Corresponding chlorophyll images.

3.2.

Reconstruction of Reflectance Image Data Using RGB Data

We computationally implemented a hyperspectral reconstruction algorithm that can reliably predict detailed spectral information from RGB data. Among several different reconstruction methods, we chose polynomial multivariate regression, described in Sec. 2.4, for reconstructing reflectance spectra for optimal performance. To generate a conversion matrix $T$ in Eq. (3), we imaged 24 A. thaliana leaf samples as follows: first, because of the relatively small sample size ($m=24$), we used the entire data set for training. To build a model for reconstructing reflectance spectra using RGB data, an RGB camera response $x$ was expressed in terms of $r$ and $S$ in Eq. (2). We obtained $S$ from the manufacturer (Sony ICX625) that was used in the handheld system [inset in Fig. 4(a)]. Second, we determined a conversion matrix $T$ by applying multivariate polynomial regression in Eq. (3). Finally, we computed a reconstructed reflectance spectrum $\widehat{r}$ for a new RGB spectrum. In this step, to determine the best polynomial degree, $x$ was varied such that $x=(1,x_1,x_2,x_3,x_1^2,x_2^2,x_3^2)$ for second-order polynomial regression, which provided reliable performance in our plant hyperspectral image data.

Fig. 4

(a) Reconstruction of full spectral information from RGB data. Representative comparison between an original reflectance spectrum obtained from the laboratory hyperspectral system and a reconstructed one from RGB data. Inset: Spectral response of the three-color CCD camera. (b) 95% confidence intervals of mean differences (dark line) between the original reflectance spectra and the estimated spectra at each wavelength. Inset: Goodness-of-fit metrics from all of the plant samples using the leave-one-out cross-validation method.

The overall performance of the reconstruction method was validated utilizing a leave-one-out cross-validation method with all of the plant samples ($m=24$). In this validation, each round of cross-validation used 23 spectra as a training set to reconstruct a full spectrum from the RGB data, and a total of 24 rounds were performed. The accuracy of the reconstructed spectral information was evaluated using goodness-of-fit metrics [i.e., coefficient of determination $R^2$ and root mean square error (RMSE)]. Given the relatively small sample size, leave-one-out cross-validation was a reasonable validation method for avoiding over-fitting. It should be noted that reconstruction models were built using only the training data set. The representative spectra in Fig. 4(a) show that a full reflectance spectrum can be reliably reconstructed from the RGB data, compared to the original hyperspectral data. To estimate errors of the hyperspectral reconstruction method, we further analyzed differences between the original reflectance spectra and the estimated spectra in the entire wavelength range. Figure 4(b) shows 95% confidence intervals for mean differences between the original reflectance spectra and the estimated spectra. As expected, the wavelength regions around 400 nm and above 700 nm have wider confidence intervals because of the limited spectral range of the RGB channel of the camera, as shown in the inset of Fig. 4(a). After including all of the samples with second-order polynomial multivariate regression, the average RMSE of cross-validation from 24 samples was 0.013 ($\min =0.001$ and $\max =0.030$) and the average adjusted $R^2$ was 0.99 ($\min =0.966$ and $\max =0.999$), as shown in the inset of Fig. 4(b). This numerical evaluation supports the feasibility that chlorophyll content can be sensitively and accurately predicted using the three-color sensor-based system.

3.3.

Spectrometerless Chlorophyll Imaging

As a pilot test, we acquired hyperspectral image data and RGB image data from Arabidopsis samples in a sequential manner. After the samples were imaged using the original laboratory imaging system (with the imaging spectrograph), the identical samples were also imaged using the handheld prototype system with the three-color CCD. Figures 5(a) and 5(b) depict representative chlorophyll images from the original hyperspectral imaging system and the prototype spectrometerless system, respectively. We compared the spatial distribution patterns of chlorophyll content $I{(x,y)}^{\text{hyperspectral}}$ and $I{(x,y)}^{\text{spectrometerless}}$ obtained from the laboratory hyperspectral system and the spectrometerless system. Specifically, we calculated a two-dimensional (2-D) correlation coefficient $C_{2\text{-}\mathrm{D}}$, which is defined as

Fig. 5

(a) and (b) Chlorophyll images generated by the laboratory hyperspectral system and the current prototype system with the three-color CCD camera using the hyperspectral reconstruction, respectively.