1.

Introduction

Imaging spectrometers collect data over three dimensions—two spatial $(x,y)$ and one spectral $(\lambda )$—so that the complete $(x,y,\lambda )$ dataset is typically referred to as a datacube. The most common method for categorizing the various types of imaging spectrometers is by the portion of the datacube collected in a single detector readout. “Whiskbroom” spectrometers, which use a linear array of detectors, collect a single column of the datacube at a time and thus scan across the two spatial dimensions of the datacube (see Fig. 1).¹ “Pushbroom” spectrometers use a 2D detector array, and thus collect a vertical slice of the datacube at once so that only one spatial dimension needs to be scanned to fill out the cube.² A filtered camera, constructed by placing a filter wheel or tunable spectral filter in front of a camera, collects a horizontal slice and thus needs to scan along the spectral dimension to complete the data set.³ Other scanning modalities exist, such as Fourier Transform imaging spectrometry (FTIS), but these can be shown^4,5 as equivalent to one of the above categories—in this case, the filtered camera.

Fig. 1

The portions of the datacube collected during a single detector integration period for (a) scanning, and (b) snapshot devices.

“Snapshot” imaging spectrometers, in contrast, collect the entire 3D datacube in a single integration period without scanning. While the existing literature cites advantages for snapshot instruments such as the lack of scanning artifacts and the increased robustness or compactness due to the lack of moving components,⁶ these qualities are actually secondary to the main benefit of snapshot collection, which has been given little attention. This is the advantage in light collection (optical throughput), which can be dramatic for larger datacubes. As a parallel to the Jacquinot (throughput) advantage and the Fellgett (multiplex) advantage nomenclature commonly used in spectrometry, we call this the snapshot advantage.

While discussion of the light collection advantages of snapshot imaging spectrometers has had some exposure in the astronomy community,^7–9 discussion has been limited to instruments coupled to astronomical telescopes. As a result, few outside the astronomy community (excepting only Refs. 10 and 11) are even aware of this important issue,¹² which has not even been given a name. We provide below the first comprehensive discussion of its characteristics across all modalities.

2.

Snapshot Advantage Factor

The snapshot advantage factor is easily derived from knowledge of the datacube dimensions and the measurement architecture. For example, for a datacube of dimensions $(N_x,N_y,N_{\lambda })=(500,500,100)$, a whiskbroom (point scanning) system sees only 100 voxels of the datacube at any given time. If the remainder of the object is emitting light during this period, then all light emitted outside these 100 voxels is lost. The overall light collection efficiency from geometric considerations alone is thus the inverse of the number of elements in the scan—in this case $1/(N_x{N}_y)=4\times 10^{-6}$. This value is cripplingly low for all but the most forgiving of experiments. For a pushbroom (line scanning) system, one sees a $500\times 100$ slice of the datacube at a given time, so the maximum full-cube efficiency value is $1/N_y=0.002$. While many experiments can tolerate such a low efficiency, dynamic scenes prevent the longer integration times needed to overcome this poor light collection. Since the $\lambda$ scan dimension in our example is one fifth that of the spatial dimensions, filtered cameras have the potential to provide a five-fold improvement in light collection ability. In practice, however, this is typically offset by light losses due to dead time between scan points or to low transmission in the spectral filters (see, for example, Ref. 13). Ignoring these losses, the geometric efficiency still remains low, at $1/N_{\lambda }=0.01$. These efficiency values given for scanning devices have been obtained by geometric considerations alone.

Not all snapshot instruments take advantage of this improvement in light collection, however. In terms of light collection capacity, one can divide snapshot techniques into two broad categories—“full-throughput” and “throughput-division” techniques—based on whether or not they sacrifice light based on their geometry. That is, although all snapshot systems remove the need to scan, and thus do not have the $1/N$ efficiency loss associated with scanning across $N$ elements, throughput-division snapshot implementations suffer from the same light collection tradeoffs as their scanning counterparts. For example, the multiaperture filtered camera^14–17 [a division of aperture (DoAp) technique, see Fig. 2(a)] consists of an array of mini-cameras each with its own spectral filter. The efficiency of each individual mini-camera, however, is reduced to $1/N_{\lambda }$ because of the bandpass filters used. A second example is the multispectral filter array camera^18,19 [a division of focal plane technique, see Fig. 2(b)]. This system uses a single monolithic lens for light collection, but places filters over each individual pixel in order to spectrally resolve light in the image. This technique thus sacrifices a $1/N_{\lambda }$ fraction in pixel fill factor, for any individual wavelength band in the datacube. The fraction of $1/N_{\lambda }$ is thus a fundamental geometric limit to light efficiency for these techniques. Due to the use of filters in both of these architectures, the light collection efficiency is thus no better than for equivalent scanning systems.

Fig. 2

System architectures for snapshot spectral techniques: (a) division of aperture (DoAp) multiaperture filtered camera, and (b) division of focal plane (DoFP) the multispectral filter array camera.

Full-throughput snapshot techniques, on the other hand, have no filters, and thus no fundamental geometric tradeoffs in light collection. There is a remarkable variety of architectures available for full-throughput imaging spectrometers, among which are (in order of provenance) computed tomographic imaging spectrometry²⁰ (CTIS), fiber-reformatting imaging spectrometry (FRIS),^21,22 integral field spectroscopy with lenslet arrays²³ (IFS-L), integral field spectroscopy with image slicing mirrors²⁴ (IFS-S), image-replicating imaging spectrometry¹¹ (IRIS), filter stack spectral decomposition²⁵ (FSSD), coded aperture snapshot spectral imaging²⁶ (CASSI), image mapping spectrometry²⁷ (IMS), and multispectral Sagnac interferometry²⁸ (MSI). See Fig. 3 for system layout diagrams. This list of full-throughput snapshot instruments is steadily increasing, and system designers can even look forward to snapshot 3D detector arrays, in which the detector itself is capable of resolving spectra at individual pixels.^29–33

Fig. 3

System architectures for CTIS, CASSI, IMS, IFS-L, fiber-reformatting imaging spectroscopy (FRIS), and filter stack spectral decomposition (FSSD).

The convergence of three recent technological advances has made snapshot imaging spectrometry possible. First is the steady decrease in cost and pixel size for large format detector arrays. These enable compact instruments with a large number of sensing elements with fast readout speed and reasonable cost. Since typical datacubes have 10 million or more elements, snapshot techniques require very large detector arrays in order to properly sample a sufficient number of datacube voxels. Only in the past decade have such detector arrays become economical. The second technological advance is in the manufacturing tools for making precision multiaperture optical elements, such as lenslet and micromirror arrays. These array optical elements allow one to design compact instruments containing a large number (up to tens of thousands) of parallel optical systems. Finally, the third technological advance, the increased computing power available to desktop computers, has enabled algorithms that can readily display and analyze the large datasets produced by these instruments.

3.

Measurements Where the Snapshot Advantage Applies

The $1/N$ values for geometric light collection efficiency relate directly to signal collection in passive measurement situations (e.g., remote sensing), in which the user has no control over the illumination source. For active illumination systems such as microscopes, however, one can compensate for a low geometric efficiency by illuminating individual pixels of the object with high intensity laser light, and measuring with a whiskbroom spectrometer. This is the method used by confocal laser scanning microscopy (CLSM). Using coherent sources to boost the illumination power density, however, faces a fundamental limit when the power becomes high enough to alter or damage the sample, or, as in fluorescence microscopy, when all fluorophores in the illuminated region have been boosted to their excited state—a situation which is largely achieved in modern confocal laser scanning microscopes.³⁴ At this point nothing further can be done on the illumination-side to increase light collection, placing a fundamental limit on overall signal. This is exactly what we have shown in a recent experiment: while the excitation laser of a CLSM excited the sample to 0.56 of the theoretical limit, the overall photon collection of the CLSM remained two orders of magnitude lower than that of an equivalent snapshot spectral imaging system, despite the use of a light source with four orders of magnitude lower power density.³⁴

An active illumination setup also allows one to encode spectral information into the illumination-side, so that the detection system need not spectrally resolve the image in order to obtain the $(x,y,\lambda )$ datacube measurement. At this point, however, we are not aware of a technique allowing this to be done without throughput loss. Rather, all techniques appear to involve either scanning³⁵ or the illumination-side equivalent of the DoAp/DoFP configurations,³⁶ so that the overall light collection suffers by a factor of $1/N_y$ or $1/N_{\lambda }$ in comparison to snapshot imaging spectrometers using broadband illumination.

For remote sensing, on the other hand, the geometric light collection efficiency is all-important. Here the user does not have the ability to manipulate the light source, and almost all object datacube voxels are continuously emitting light, so that only a parallel light collection technique can capture the full signal. For scanning instruments, this setup results in a tradeoff between light efficiency and the number of scan elements, a feature which has frustrated the expansion of imaging spectrometry into new fields where there is just not enough light to permit a tradeoff. These include, for example, spectral imaging of dynamic objects, target tracking,³⁷ and overcoming signal-to-noise-ratio-limited spectral unmixing.^38–40

The full-throughput snapshot advantage does, however, come at the price of an increase in system complexity, either in the optical hardware or in the reconstruction software. Most of the snapshot techniques involve arrays of optical elements, and thus require advanced manufacturing techniques that have only recently become available. In addition, with the exception of CASSI, all of these instruments require large format detector arrays, and this is perhaps their primary limitation. Detector technology, however, has been advancing at a pace paralleling that of Moore’s law,^41,42 so that we can expect these limitations to ease in the coming years, both in terms of the overall pixel count, cost per pixel, and pixel readout speed.

One may argue that the complexity tradeoff compromises the snapshot advantage. The division of aperture technique, for example, consists of an array of mini-cameras each with its own spectral filter. For an array of 25 cameras [as shown in Fig. 2(a)], the system pupil is 25 times as large as the pupil of each individual camera. Thus, if we compare a full-throughput technique with a DoAp, we can say that the simplicity of the DoAp should allow one to implement a larger pupil than the snapshot technique can, and this should improve light collection. A similar argument holds for the multispectral filter array camera: using focal plane division techniques allows one to use front end optics with lower resolution than a comparable snapshot system, and this resolution change can be achieved simply by increasing the pupil diameter, which improves on light collection.* In practice, however, the tradeoff between complexity and light collection has not significantly impacted instruments presented within the journal literature: the DoAp and DoFP approaches so far constructed (see Refs. 14, 16, and 19) do not show an order of magnitude larger pupil diameters than their full-throughput counterparts have been able to achieve (see Ref. 27).

Note that although CASSI, IRIS, and MSI all suffer from a 50% efficiency loss (the first due to the use of a binary mask, the others due to the need to polarize incoming light), these are still labeled as “full-throughput” techniques because the factor of two in light lost will be much lower than the factor of $N$ advantage due to snapshot collection. CTIS also suffers from significant light loss due to inefficiencies in grating dispersion into the designed diffractive orders, but this factor will also generally be small in comparison to $N$. Finally, while one advantage of snapshot instruments is the absence of scanning artifacts when imaging moving objects, this does not imply that one obtains the full data in real time. Both CTIS and CASSI are computationally intensive instruments, and this can create a considerable delay between raw data acquisition and the final delivery of the datacube. An overview of the various snapshot instruments and their maximum theoretical efficiency values are given in Table 1.

Table 1

Snapshot instruments and their maximum theoretical efficiency values.

4.

Snapshot High-D Systems

The snapshot advantage in imaging spectrometry is a direct analogue of the advantage of staring versus scanning infrared imagers demonstrated during the 1980s and 1990s.^44–47 Scanning infrared imaging systems used single-point detectors scanned in two dimensions across a scene, or a linear array of detector elements scanned across one dimension of the scene, in order to obtain a complete 2D image. Scanning systems suffered an efficiency loss equal to the number of elements in the scan dimension as a direct result of using a lower-dimensional detector array (single detector or 1D array) to measure a higher-dimensional dataset, the 2D image. This is equivalent to the imaging spectrometer problem of detecting a 3D dataset on a lower-dimensional 2D detector array. While infrared detectors evolved to allow detectors whose dimensionality matched the measurement data (2D for an image), the only way for an imaging spectrometer to avoid scanning is to design an optical system in which the light distribution on the 2D detector array encodes the full three-dimensional distribution of light within the object’s datacube. Doing this encoding without sacrificing light achieves the snapshot advantage.

The concept of a snapshot advantage also extends beyond just imaging spectrometry. It applies equally well to any high-dimensional (high-D) system—an instrument whose data dimensionality is higher than just the two dimensions available for detector arrays. The plenoptic function $I(x,y,z,{\theta }_x,{\theta }_y,\lambda ,s,t)$ describes the complete distribution of data obtainable from passively sampling the optical field,⁴⁸ and thus describes the highest data dimensionality to which we have ready access via optics. (Here $s$ and $t$ describe the polarization and time variation of the optical field.†) Since higher-dimensional measurement systems parcel the finite number of photons collected into ever smaller bins, maintaining snapshot capability becomes important for anything beyond the measurement of static objects in a laboratory setting.

The “light field camera,” for example, is a snapshot instrument which collects angularly resolved image data $I(x,y,{\theta }_x,{\theta }_y)$ by re-mapping the 4D distribution onto a two-dimensional detector array.⁴⁹ A similar but much less compact implementation uses an array of individual cameras.⁵⁰ These snapshot aprroaches thus have a $N_{{\theta }_x}\times N_{{\theta }_y}$ throughput advantage over any system which scans over angle in order to obtain the full dataset.‡ This is separate from the reduced signal-to-noise ratio in each data element due to the use of smaller bins that come with higher dimensionality measurement. Snapshot techniques thus become increasingly important with increasing dimensionality, with the tradeoff that much larger detector arrays are needed to accommodate the larger datasets.

Other examples of snapshot high-D systems include channeled imaging polarimeters,^51–53 which measure an $I(x,y,s)$ dataset; line imaging spectropolarimeters^54,55 which measure $I(x,\lambda ,s)$; and computed tomographic imaging channeled spectropolarimeters^56,57 (CTICS), which measure $I(x,y,\lambda ,s)$. For polarization systems, the snapshot advantage in light efficiency is limited, since the theoretical maximum efficiency improvement over a scanning system is only 4 (for a Stokes polarimeter) or 16 (for a Mueller matrix polarimeter). Since polarimetry typically requires computational reconstruction of the data, the need for accurate calibration⁵⁸ means that snapshot systems’ lack of moving parts is usually the more important feature.

5.

Conclusion

When measuring high-D data, full-throughput snapshot instruments have a light collection capacity which exceeds that of all scanning and all throughput-division snapshot instruments by a simple geometric factor which we call the snapshot advantage. Any experimental setup whose measurement dimensionality exceeds that of the detector, and in which all data elements (e.g., datacube voxels in imaging spectrometry) are luminous throughout the measurement period can use this advantage fully. While there currently exist only a handful of instruments capable of full-throughput snapshot measurements of 3D or 4D data, we expect to see more as researchers find new ways of adapting new technology to these challenging measurements.

Since the full-throughput snapshot techniques map each element in the data to an individual pixel, the primary limitation to constructing snapshot versions of such instruments is the limited number of pixels available with current detector arrays, such that any system attempting to perform snapshot measurements beyond 4D will need to wait for the development of much larger detector arrays. At some point instrument designers may learn how to relax this “curse of dimensionality”⁵⁹ by taking advantage of ideas such as compressive sensing,⁶⁰ but we have not yet learned to do this while maintaining data fidelity.