2.1.

NIR-HSI System

The NIR-HSI system, Hyperspec NIR XS-100 (Headwall Photonics, Fitchburg, Massachusetts), consists of a 14-bit InGaAs charge-coupled device line array detector, a push broom imaging spectrograph, a standard $f/2.0$ C-mount lens, an illumination unit of two 180-W tungsten halogen lamps with a slit (18 mm in length, $25\mu \mathrm{m}$ in width), and a sample transport mechanism. The detector captures reflected light from samples line by line in the NIR spectral range from 900 to 1700 nm, in which there are totally 168 bands in the case of $\sim 5\text{-}\mathrm{nm}$ intervals. The spatial resolution of the system is about 0.8 mm/pixel. The image acquisition software is Hyperspec^® (Headwall Photonics, Fitchburg, Massachusetts).

2.2.

Image Acquisition

Gastric cancer specimens were collected immediately from randomly selected patients who underwent partial gastric resection for tumor removal at People’s Hospital of Huangpi District, Wuhan, China. A total of 29 samples, 17 males and 12 females, the mean age of the patients was 54 years with the oldest 80 years and the youngest 26 years. All specimens, mucosa surface upward, were kept on a black plastic plane. Surface moisture was wiped by absorbent paper before image acquisition at room temperature. To acquire hyperspectral images, the transport mechanism was moved at a constant speed of 2 mm/s. After measuring and marking the tissue surface, these specimens then were sent to H&E. The study was approved by the local ethics committee, and informed consent for use of samples was obtained from all patients.

2.3.

Image Processing

Image processing, spectral extraction and analysis were carried out by the Environment for Visualizing Images (ENVI 4.7) software (Research Systems Inc., Boulder, Colorado). In addition, the scientific graphing software Origin 8.0 (OriginLab, Northampton, Massachusetts) was also used for spectral analysis.

To eliminate the influence of the dark current and the nonuniformity of the illumination device, the raw hyperspectral image should be calibrated and normalized. A standard reference white panel was placed in the scene of imaging and its data were utilized as the white reference. The reflectance from the white panel provides an estimate of the incident light on the tissues at each wavelength. The dark current was captured by turning off the light source along with completely covering the lens of the detector with its opaque cap and recording the detector response. Then the data were normalized to find a relative reflectance using the equation.⁶

After calibration and normalization, a mask was created to remove background noise at 1181 nm. A mask image is a binary one that consists of values of 0 and 1. When a mask image is used in a processing function, ENVI includes the areas with values of 1 and ignores the masked 0 values in the calculations. Only the masked image was subjected to further data analysis.

2.4.

Spectrum Extraction and Analysis

The H&E result as the “gold standard” was applied to extract characteristic spectra of cancerous and normal tissues. The region of interest (ROI) of gastric cancer was manually selected in a zoom window containing the cancerous areas using the circle drawing mode provided by ENVI ROI tool. A small circle (4 pixels) was first drawn within a selected lesion, and the circle was then grown to merge the neighboring diseased pixels using a specified standard deviation (20%) away from the mean of the drawn region. The value of the standard deviation multiplier was determined by visual inspection of the increased ROI in such a way that all the pixels in the grown area were cancerous and the edge pixels were also avoided at the same time. This “grow” ROI selection method is especially effective for selecting the diseased areas with irregular shapes.¹⁶ The cancerous mean spectrum was calculated among all diseased pixels. The mean spectrum of normal tissue was extracted by the same flowchart and approach.

2.5.

Cancer Diagnosis

Since hyperspectral images have large amounts of hypercube data containing spectral and spatial information, dimensionality reduction is one of the most important steps during the spectral analysis. To reduce the dimension of the hyperspectral image, PCA was employed to extract a set of orthogonal principal components (PCs) comprising scores and loadings that account for the maximum variance in spectral datasets.¹⁷ The scores of PCA represent the weighted sums of the original variables without significant loss of useful information, and the loadings can be used to identify important variables that are responsible for the specific features appeared in the corresponding scores.¹⁸ The optimal wavelengths were selected based on the maximum and minimum loadings of the PCs with highest weights, which contributed most spectral variance between cancerous and normal.

The selected optimal wavelengths were used to replace the full wavelengths for diagnosis with SAM. SAM is a supervised classification method that allows rapid mapping of the degree of similarity between image spectra and reference spectra. Smaller angles represent closer matches to the reference spectra.¹⁵ This simple classification tool is often used as a first approach to the hyperspectral data and reliable when images have brightness shifts and other spectral artifacts present compare to other classification algorithms. The appropriate reference spectra were selected from five training samples according to H&E to train SAM classifier. Then, the trained SAM classifier was used to diagnose 24 new samples for model verification, and histopathological results served as the “gold standard” for assessment of the diagnostic effect of HSI technique.

3.1.

Hyperspectral Images

A hyperspectral image, known as a hypercube, contains three-dimensional block data that provide spatial information along with spectral information for each pixel in each image, as shown in Fig. 1(a). A hyperspectral image is made up of 168 contiguous wavebands for each pixel. Each pixel in the hyperspectral image has a sequence of intensities in different wavelengths, as shown in Fig. 1(b), which constructs the spectral signature of that pixel. The resulting spectrum acts like a fingerprint, reflecting the composition of a certain pixel. The difference in spectral signature between the cancerous and the normal tissues can be distinguished. Hyperspectral images allow for the visualization of biochemical constituents of each pixel, separated into particular areas since similar spectral properties have similar chemical composition.¹⁸

Fig. 1

HSI hypercube diagram. (a) Hypercube data including spectral and spatial dimensions. (b) Reflectance spectrum of the pixel ($i,j$) (red dot).

3.2.

Characteristic Spectra

Figure 2 shows mean spectra and standard deviation of the reflectance of the pixel from the normal and cancerous regions. Red solid and blue lines represent the mean spectra of cancerous and normal pixels, respectively. The rest are the standard deviations that are nearly constant during the whole spectral range, which suggests the spectra are highly reproducible.

Fig. 2

Mean spectrum (red dash line) and standard deviation (green dash lines) of cancerous pixels, mean spectrum (blue solid line) and standard deviation (yellow solid lines) of normal pixels.

Comparing the mean spectra, a distinctly observed feature useful for recognition of cancerous and normal tissues is that the reflectance level of cancerous tissue is higher than those of normal tissue during the whole wavelength region. This is probably due to the physical surface properties, such as color, shape, and texture. A similar result was observed by Liu et al.¹⁹ detecting tongue cancer in medical hyperspectral images. Cancerous and normal tissues show different spectral shapes according to their different chemical structure and composition. The most prominent absorption bands occurring in the NIR region are related to overtones and combinations of fundamental vibrations of C-H, N-H, O-H, and S-H functional groups.²⁰

According to Fig. 2, the spectral difference can be clearly identified in three regions (950 to 1050 nm, 1150 to 1250 nm, and 1400 to 1500 nm). By inspecting cancerous mean spectra, it was found that the major absorption peaks were observed around 975, 1215, and 1450 nm. The most intensive absorption band around 1450 nm is related to O-H stretching first overtone.²¹ The moderate absorption band around 1215 nm is attributed to C-H stretching second overtone.²² The weak absorption band around 975 nm is assigned to O-H stretching second overtone.²¹ The whole spectrum is a mixture of the spectral signature of many tissue components, including water, lipids, proteins, and carbohydrates.^23,24

Yi et al. have investigated the application of near-infrared (NIR) spectroscopy equipped with a fiber-optic probe for differentiation gastric cancer. Major spectral differences were observed in the C-H stretching first overtone, combination band, and second overtone regions.²⁵ By the use of unsupervised pattern recognition, all spectra were classified into cancerous and normal tissue groups with high accuracy. Similar discrepancy of NIR spectrum for the diagnosis of pancreatic and colorectal cancer has been reported by Kondepati et al.^23,24 The similarity indicates the coherence of carcinoma differentiation by NIR. These results show the high discriminating power of the NIR spectrum extraction from hyperspectral image in the identification of cancerous and normal tissue spectral attributes.

3.3.

PCA and Optimal Wavelengths

Figure 3(a) shows the PC eigenvalue plot. The horizontal axis represents the eigenvalue number and the vertical axis represents the eigenvalue. The number of PC can equal the score of bands in the original images; however, only the first few PC contain the majority of uncorrelated information.¹⁴ The first PC contains the largest percentage of data variance, the second PC contains the second largest percentage of data variance, and so on. The last PC appears noisy because it contains very little variance, much of which is due to the noise in the original spectral data. The relative weights of PCs were calculated by dividing their eigenvalues in the eigenvector. PC1 and PC2 explain 94.22% and 3.28% of original variance, respectively.

Fig. 3

PCA and optimal wavelengths: (a) PC eigenvalue plot and (b) a total of six wavelengths (i.e., 975, 1075, 1215, 1275, 1390, and 1450 nm) selected as optimal wavelengths.

From the plot of wavelengths versus the loadings of the first and second PCs, it was decided to select those wavelengths situated at the maxima or minima at each plot, as shown in Fig. 3(b). Five wavelengths (i.e., 975, 1075, 1215, 1275, and 1450 nm) were selected from PC1 and only one wavelength (i.e., 1390 nm) was selected from PC2. A total of six wavelengths were then extracted as optimal wavelengths that can be used to discriminate cancer from normal.¹⁸ These optimal wavelengths not only reflect the physical/chemical information but also maintain the robust diagnosis and classification efficiency.²⁶

In this case, PCA was employed to reduce the dimensionality of the datasets from 168 spectral dimensions to only six dimensions. Moreover, these optimal wavelengths, which later may be implemented in a real-time multispectral imaging system, will decrease image acquisition and processing time significantly.¹⁸

3.4.

SAM and Supervised Classification

Figure 4 shows some selected images, which comprise the conventional RGB images [Fig. 4(a)], ROI images [Fig. 4(b)], binary images [Fig. 4(c)], masked images [Fig. 4(d)], and gastric cancer diagnosis by training SAM [Fig. 4(e)].

Fig. 4

Five representative images of gastric tissues for model training, comprising (a) conventional RGB images, (b) ROI images selected by ENVI ROI tool, (c) binary images, (d) masked images, and (e) gastric cancer diagnosis result by training SAM.

Figure 4(a) shows the conventional RGB image of gastric tissues with impalpable distinction in color. It is not a small challenge for clinicians to distinguish where it is normal and abnormal using conventional reflectance endoscope.⁴ Figures 4(b)–4(d) show the processing results of hyperspectral images for spectral extract and analysis. Figure 4(e) shows the diagnosis result image with training SAM. The red regions are cancerous pixels and the rest are normal ones in Fig. 4(e). The classification result is clear and obvious, which offers great assistance to clinicians.

Figure 5 shows the conventional RGB image [Fig. 5(a)] and gastric cancer diagnosis result by trained SAM [Fig. 5(b)] for 24 new samples. In Fig. 5(b), cancerous regions were colored with red and the normal ones were colored with blue. For assessment of the effect of HSI technique, histopathological results served as the “gold standard.” Accuracy is given by the ratio of (TP + TN)/(TP + FP + TN + FN), where TP and FN are the number of the true positive (cancer) and false negative results and TN and FP are the number of true negative (normal) and false positive results, respectively.²⁷ While comparing the classification results in Fig. 5(b) with H&E results, the accuracy is up to 90%, which testifies SAM classification algorithms are relatively effective for cancer diagnosis with HSI.⁴ The diagnostic effect with SAM is better than other algorithms’ according to the accuracy,^7,8 because SAM is insensitive to illumination and albedo effects, spectral angle will be relatively insensitive to changes in pixel illumination, increasing or decreasing illumination does not change the direction of the vector, only its magnitude.²⁸

Fig. 5

Representative images of gastric tissues by trained model: (a) conventional RGB images and (b) classification map (cancerous regions colored with red and the normal ones colored with blue) with trained SAM for new specimens.

On one hand, using NIR (900 to 1700 nm), including tissue window²⁹ as working wavelengths instead of visible-light region (400 to 800 nm),^6–8 will improve the tissue penetration depth and get more diagnostic information,⁵ which could detect lesions in the submucosa of the tissue. On the other hand, using the six optimal wavelengths as input endmember have tremendously decreased the spectral processing time nearly up to real-time requirement with relatively high accuracy. Furthermore, this work and other studies^6–10 using HSI without time-consuming stains show that HSI technology may provide a new tool for histological analysis, which could improve both morphometric and biochemical analysis at the same time.