1.1

Instrument concept

The instrument concept is centered around the computational spectrum inversion using compressive sensing. Compressive sensing is a signal processing technique for efficiently acquiring and reconstructing a signal. It can find a solution to an underdetermined linear system where the number of measurement (M) is smaller than the sampling points (N) of the to be determined signal [4]. In general, this is not possible as stated by the Nyquist sampling theorem. However, it can be achieved if the signal is sufficiently sparse in some base or representation. The prior knowledge of this sparsity allows reconstruction algorithms to find solutions to the underdetermined linear systems which approximate the underlying signal very well. The under sampling enables many types of advantages, such as reduced data flow, improved signal-to-noise ratio (SNR) or increased data acquisition speed. For these reasons compressive sensing is currently used in many applications, such as imaging, pattern recognition and video processing [4]. Because absorption bands of trace gases are highly periodic, radiance spectra of the earth can be effectively described in a Fourier base to create a sparse signal. Using compressive sensing it is possible to therefore reconstruct a detailed spectrum with fewer measurements (M < N). Such an approach would result in a high SNR compared to a linear gradient filter, allowing an uncompromised compact instrument with high resolution and high spectral accuracy.

Figure 1.

Instrument concept. a) The instrument uses the push broom principle the measure the intensity transmitted by each filter. b) The spectral filters cover the pixels of the focal plane array in a column wise fashion, covering multiple pixels. c) Principle of compressive sensing. Here a solution to signal S (Nx1) is found with Y (Mx1) measurements. D) Each spectral filter has a unique spectral transmission profile.

Here we propose the following instrument concept, which has similarities with a linear gradient filter spectrometer, but making use of compressive sensing, see Figure 1. The instrument orbits the earth and consists of an earth observing telescope. A wafer with a dielectric material (in our case a glass wafer with a 400 nm thin Si-film) is bonded on the sensor. It contains 2D photonic crystals in a column wise arrangement in the across-track (ACT) direction. These photonic crystals consist of sub-wavelength features etched in the Si-film and have unique spectral transmission profiles. These transmission profiles arise from resonances in the Si-film and depend on the thickness of the film as well as the specific shape and size of the etched structure and the lattice size. By choosing a specific hole shape and lattice size, different spectral transmission profiles can be achieved. Using the push-broom concept, the transmitted intensity for each ground instantaneous field of view pixel can be measured for each spectral filter. This allows a far larger field-of-view in along-track (ALT) direction than with an imaging spectrometer with a slit. As only a limited number of filters is needed to reconstruct the spectrum, the filters can cover multiple pixels of the sensor. Imaging in ACT direction follows the conventional push-broom concept. From the transmission measurements in ALT direction of the same ground scene an input spectrum can be reconstructed.

1.2

Conceptual advantages and disadvantages

The instrument concept presented here features several advantages over existing implementations. It is smaller than a grating spectrometer since no additional free space optical path length is needed for wavelength separation. Moreover, different spectral ranges can be combined on one detector (for example the O2A band and methane band, or several relevant methane spectral ranges). The concept is similar to the linear gradient filter concept, but it features a higher spectral bandwidth per filter and therefore has a much higher SNR. This is in turn amplified as the spectrum reconstruction using compressive sensing potentially allows to decrease the number of required filters, which further increases the SNR. However, we expect that the performance of such an instrument might suffer in fast changing scenes and in addition will require thorough calibration.

2.1

Computational spectrum reconstruction

The measured intensity I_m of filter m can be described as

where Sⁱⁿ (λ) is the input signal, D(λ) the detector response and T_m (λ) the transmission function of filter m. Alternatively, with λ discretized and the detector response neglected, eq. 1 can be reformulated as a system of equations and stated as a compressive sensing problem

where M is the total numbers of filters with N > M. Here T denotes a matrix containing the transmissions of the spectral filters. Each row of T corresponds to the spectral transmission of one filter. Because N > M, eq. 2 does not have a unique solution in general. A unique solution to the problem of eq. 2 can still be found by minimizing the L1-norm of the reconstructed signal in a sparse base, here assumed to be Fourier space

where ϕ^–1 is the inverse Fourier transform matrix and the Fourier transformed signal. However, finding this solution in practice is computationally intense. Alternatively, the problem can be regularized as follows

where α denotes the regularization factor. Also called penalty coefficient, α in eq. 4 can be interpreted as the sparsity constraint on the solution. For the limit where α approaches zero, the residue term dominates, and the problem effectively reduces to an energy minimization problem. In the limit where α approaches infinite, the length term dominates and solutions for S including many non-zeros are penalized. A balance can be found where a specific sparsity can be retrieved for a specific α [5]. Here we use the LASSO algorithm, which is a standard optimization algorithm for compressive sensing, to find a solution of eq. 4. Alternatively, Ordinary Least Squares (OLS) can be used in combination with gradient decent to find a solution with α = 0.

2.2

Photonic crystal based spectral filters

The compressive sensing algorithm can reconstruct the unknown signal as best as possible if the transmission profiles of the photonic crystals are highly uncorrelated [6]. To obtain such uncorrelated set of filters we first created a library of photonics crystals in a wide variety of shapes: squares, circles, octagons, and two types of crosses in a range of different sizes (w) and lattice constants (Lc). See Figure 2. We used finite-difference time-domain (FDTD) electromagnetic simulations in Lumerical (version 3.8, 2022) to compute the spectral transmissions. The relevant simulation settings used are: pulse length of 15 ps and a mesh accuracy of 2 (mesh size depended on refractive index of the material, 6 spatial samplings per wavelength). We varied the lattice constant between [600,2000] nm with steps of 20 nm and the width of the structures ranging from 0.25Lc to 0.65Lc in 5 steps. After some initial filtering to remove uninteresting configurations we ended up with a library of around 1300 filters.

After the library generation a set of filters with a low amount of correlation must be obtained. The three steps of this filter selection procedure are as follows. First, 700 filters were iteratively removed using singular-value decomposition (SVD). For this the row and column wise correlations are computed to perform the singular value decomposition. The filter which best resembles the most common mode is removed and the process if repeated until 600 filters were left. In the second step of the filter selection the pairwise correlations of the remaining filters are ranked and the filter of the pair with the highest correlation is removed. The filter with the least amount of correlation with the rest of the filter set is kept, the other filter is discarded. This process is repeated until the desired number of filters is achieved.

Figure 2.

Illustration of the FDTD simulation and the investigated shapes. A&B) 3D rendering of the simulation domain. Grey: SiO2 substrate; red: Si-film; white: etched structure; orange: simulation domain incl Perfectly matched boundary (PML) layer.

The acquired filter set is now highly uncorrelated, but there is a chance that uncorrelated filters have been removed in step 1. Therefore, the last step of obtaining an uncorrelated filter set involves re-introducing discarded, but potentially better filters. Potential candidate filters are identified by comparing the correlations between the current set of filters and the removed filters. If a removed filter has a smaller correlation with respect to all filters in the current set, it is replaced with one of the two filters that has the largest correlation. The choice of filter between these two depends on their respective L2-norm of the overall correlation with the complete set.

Using this procedure, we obtained several sets of filters with a different total number of filters ranging from 200 to 40. A reduced amount of filters is in principle favorable because this increases the SNR as each filter can cover more pixels. However, less filters might decrease the spectral reconstruction quality due to a lower amount of input information.

2.3

Input methane radiance spectrum

To test the signal reconstruction of the compressive sensing algorithm an input spectrum is required which act as ground truth signal for the signal reconstruction algorithm. We simulated an earth radiance spectrum using a non-scattering radiative transfer model for the atmosphere, assuming Lambertian reflectance at the surface of the earth. The relevant molecule absorption bands are obtained from the HITRAN database [7] and we assumed a methane concentration of 2000 ppb. The data set is simulated with a non-linear sampling of 0.005 nm in a spectral range of 1625 nm to 1670 nm for an albedo of 0.15, sun zenith angle of 70 degrees and nadir observation. Here we focus on the radiance between 1625 nm and 1670 nm where periodic methane absorption bands present. This spectrum act as the ground truth signal and allows the quantification of the reconstruction error of the compressive sensing algorithm.

Figure 3.

Comparison between ground truth and reconstructed spectrum for different sized filer sets A-C) reconstruction with 200 (A), 120 (B) and 40 (C) filters used. D-F) Fourier transform of the ground truth signal and reconstructed signal for (A-C).

3.1

Experimentally measured filter transmission

To demonstrate the experimentally viability we have manufactured a test proof-of-principle spectral filter using e-beam lithography and measured the spectral transmission of the photonic crystal, see Figure 4. The spectral transmission was measured using a setup which contained a tunable laser in the range of 1530-1610 nm (Newport, custom version of TLB-6730-P). The laser emission was collimated using a reflective fiber collimator (Thorlabs, RC04FC-P01) and directed to the wafer. The small etched region was then imaged by a 4X imaging relay onto a InGaAs sensor (Xenics, Xena640). Etalon effects of the wafer were removed by a fitting algorithm.

The measured transmission, shown in Figure 6c is in good correspondence with the simulated transmission, thereby validating our filter simulation framework. However, the spectral feature around 1540 nm is not as sharp as the spectral feature at 1568 nm. There also appears a slight overall difference in transmission. Both these differences between simulation and measured transmission are attributed to manufacturing precision and tolerances, such as the precise thickness of the Si-film and defects in the etching processes. These results show that it is possible to create very sharp spectral features, but also indicate that the lithography protocol needs to be further optimized to achieve a consistent performance of these photonic crystals.