Predicted target acquisition range depends upon a variety of parameters such as system noise, system modulation transfer function (MTF), atmospheric conditions, and target characteristics. The analyst must understand the underlying physics to interpret program outputs and be able to distinguish the effects of each parameter. Models such as the night vision integrated performance model (NV-IPM),1* provide a variety of useful trades. These include range probability versus focal length, aperture diameter, or jitter. However, these trades frequently do not provide insight into basic camera design.
There are many variables. What should be kept constant for a trade study? Choices are field-of-view (FOV), number of detectors (), detector size, , chip size, focal length, , aperture diameter, , and -number, . Many are linked such as and . Each graph looks quite different and represents a different slice though this multidimensional space. Optimization of any one parameter becomes more difficult as atmospheric transmission, turbulence, target characteristics, and task difficulty are added.
Range performance depends upon both the optical blur, which is proportional to , and detector size. The approach2–4 appears applicable to all imaging systems. Small values represent detector-limited operation, whereas, large values represent optics-limited operation which is also called the diffraction-limited region. In the optics-limited region, decreasing the detector size does not generally increase range. The suggested minimum detector size occurs when the detector size no longer influences range performance. From a sampling view point, this occurs when .
There are an infinite number of combinations that satisfy . Using a system design approach, this paper suggests that the smallest practical detector size is 2 μm for the mid-wave infrared region (MWIR) and 5 μm for the long-wave infrared region (LWIR) with optics when the signal-to-noise ratio is high. For challenging signal-to-noise ratio conditions and more reasonable optics, a 3 μm MWIR detector, and a 6 μm LWIR detector are recommended. Our approach provides the smallest volume, lowest weight imaging sensor.
In keeping with the current U.S. military mixed unit terminology, the units of focal length and aperture diameter are inches, FOV is degrees, detector size in micrometers, chip size in millimeters, background temperature in Celsius, and the noise equivalent differential temperature (NEDT) in milliKelvin. In this paper, the FOV, , and spatial frequency refer to the horizontal direction only. The same equations apply to the vertical direction. The small angle approximation is used throughout the paper although it is not accurate when the FOV is .
Section 2 addresses the rational for small detectors. Using a step-by-step approach, the MTF is described in Sec. 3 with object space units of cycles/mrad. Then sampling is added to illustrate aliasing. As described in Sec. 4, the most important component is the reconstruction filter. Its presence creates the spurious response (SR). Section 5 introduces a simple figure-of-merit (FOM) from which the acquisition range can be calculated. The NEDT is discussed in Sec. 6. Section 7 provides practical aspects that suggest one wavelength band may be a better choice than the other. As discussed in Sec. 8, final size depends upon optical design difficulty, manufacturing constraints, noise, and the scenario such as cold weather operation, atmospheric conditions, and task on hand. The precise values, shown in this paper, are unique to the spectral responses selected. However, the results are sufficiently general to apply to all imaging systems.
This paper is an extension of Ref. 4. It provides more detail and, thereby, has broader applicability to all imaging systems.
Why Smaller Detectors?
Over the past few years, significant progress has been achieved in small detector fabrication. A few companies have fabricated 5 μm MWIR5 or LWIR detectors in test devices. Fixed format array is the most popular design and are typically , , , or pixels. When the detector format is selected, a smaller detector corresponds to a smaller chip. This lowers the detector cost since a single wafer can yield more focal planes. This leads to economy of scale.
In the detector-limited region, reducing the detector size provides better resolution if the focal length is kept constant. This leads to the popular statement, “Smaller detectors provide better resolution.” But the array size is also reduced leading to a smaller FOV. On the other hand, if the focal length is reduced in the same proportion as the detector size, the instantaneous-field-of-view (IFOV), and FOV remain the same. In the optics-limited region, smaller detectors have no effect on system spatial resolution.
For a fixed FOV, fixed -number, and detector-limited system, a smaller detector mandates a smaller focal length and aperture size. Here, the optics volume decreases with the cube of the detector size. In the optics-limited region, a smaller detector decreases the optics volume by the focal length providing a linear reduction in optical volume. In the transition region between detector-limited and optics-limited, the optics volume reduction is between a cubic and linear reduction. Commensurate with reduced volume is reduced weight. This process leads to size, weight and power (SWAP) reduction.6
Even more significant than system size reduction, is the potential reduction in optical complexity. For example, consider a large array consisting of 5 μm detectors. Coupled with a 6-inch aperture and optics, the array provides a 9.4 deg wide FOV which is suitable for search and detection. By selecting the central detectors, the FOV narrows to 0.96 deg, which is ideal for long range target identification. The FOV switch is performed in software. In fact, the change in FOV is continuous from wide-field-of-view (WFOV) to narrow-field-of-view (NFOV). Both the detection and identification functions exist with a single, low -number optical system.
MTF and Aliased Signal
The system MTF is dominated by the optics, detector, and display MTFs. Since the detector spatially samples the scene, sampling artifacts further corrupt the image. Other blurs such as aberrations, crosstalk, diffusion, and antenna effects are considered negligible in our approach.
The MTF for a circular diffraction-limited optical system is
The object-space cutoff frequency is , where is the aperture diameter and has units of cycles/mrad. The average wavelength is used: for the MWIR and for the LWIR. The detector MTF is
The detectors are square with each side . The IFOV, , is simply the detector size divided by the effective focal length: . The detector MTF exists for all frequencies from to . The detector cutoff is defined as the first zero in Eq. (2). It occurs when . For simplicity, the detector MTF is plotted up to its cutoff frequency only.
Considering the optics and detector combination, is the ratio of the detector cutoff frequency to optics cutoff frequency. In space, is a measure of the optical blur diameter relative to the detector size. uniquely defines the shape of the optics/detector MTF combination.
The sampling frequency is determined by the detector pitch. Assuming a 100 percent fill factor array, the pitch is equal to the detector size. The sampling frequency, , is equal to the detector cutoff and the Nyquist frequency, , is . In the frequency domain, the scene spatial frequencies are replicated about integer multiples of the sampling frequency. For mathematical simplicity and simple graphs, only the first replication is shown. The higher order replicas further distort the imagery. While not shown, they do not significantly alter the conclusions drawn here. In Figs. 1 and 2, the detector cutoff frequency is normalized to unity and the optics MTF changes with . . As will be shown later, the detector-limited region occurs when and the optics-limited region occurs when .
In-band refers to all spatial frequencies from zero to the Nyquist frequency. The in-band replicated signal decreases as increases. As illustrated in Fig. 2, it is zero when . This suggests image quality will not be corrupted by sampling artifacts in the optics-limited region. This is only true if an ideal reconstruction filter is present. The ideal filter has unity response up to the Nyquist frequency and zero thereafter.
Digital data cannot be seen because it resides in a computer. The reconstruction filter converts it into viewable analog data. Cathode ray tubes (CRT), flat panel displays, light-emitting diode (LED) projectors, and printers are all reconstruction filters. Each has its own unique frequency response. None are ideal. CRT-based displays act somewhat like an ideal reconstruction filter. The work by Schade, Legault, Sequin, and others researched CRT frequency response. They tried to minimize the MTF at Nyquist frequency while maintaining a high in-band response.
A flat panel display does not have the same frequency response as a CRT. Each monochrome display element provides the output of one detector. Here, the display MTF is identical to the detector MTF.
As illustrated in Figs. 3 and 4, the flat panel display modifies both and the replicated signal. Here, . The modified replicated signal is the SR. The SR represents frequencies that were not in the original scene and it reduces image quality. As illustrated in Fig. 4, there is some out-of-band SR when . Out-of-band frequencies are those greater than the Nyquist frequency. Higher values will further reduce the SR. However, it will also reduce in-band and further reduce contrast and edge sharpness.
The human visual system (HVS) is another reconstruction filter. For convenience, the HVS response is defined as the inverse of the contrast sensitivity function, , normalized to unity. As illustrated in Fig. 5, as the observer moves away from the display he will no longer be able to perceive the higher frequencies. Therefore, the SR in Fig. 4 is considered minimal in terms of perceived image quality.
NVThermIP, the precursor to NV-IPM, plots two contrast transfer functions (CTF)
The definition of the variables can be found in Ref. 7. Although includes the eye CTF, the two components appear similar in shape when the three-dimensional noise components are zero. This suggests that the frequency component of is dominated by . Assuming , the approximations are
The integral in the targeting task performance (TTP) is our FOM. It does not include the NEDT which is discussed in Sec. 6. The lower limit of integration is near zero. For high contrast targets, the upper limit is the Nyquist frequency. Then
The integral3 was numerically evaluated for a variety of values normalized to the optical cutoff. NVThermIP was run at a variety of values to obtain the out-of-band SR. These values were multiplied to create a FOM that includes the SR. As illustrated in Fig. 6, a curve fit valid up to is
There are two distinct operating regions. They are detector-limited when and optics-limited when . The transition region is large. When , the Airy disk is equal to the detector size. The MTFs are illustrated in Figs. 1 and 3. When , the optics MTF is zero at , as illustrated in Figs. 2 and 4. The location of the optics-limited region is somewhat arbitrary. Schade’s equivalent resolution and the modulation transfer function area (MTFA) suggest3 that the optics-limited region occurs when .
NVThermIP documentation7 defines inherent contrast as the target, , divided by the scene contrast, . The apparent target contrast, , is the target inherent contrast modified by the atmospheric transmittance, :
Neglecting the NEDT, acquisition range depends upon the target effective size, W, and task difficulty V:
The advantage of the FOM is that it is a system characteristic that is independent of noise and atmospheric conditions. Since the range is proportional to the FOM, it is convenient to plot relative range versus as illustrated in Fig. 7. When detector-limited, decreasing the detector size has a dramatic effect on range. On the other hand, in the optics-limited region, decreasing the detector size has minimal effect on range performance.
In the detector-limited region, range is independent of wavelength and inversely proportional to the IFOV. Since the FOV simply depends upon the number of detectors, range is inversely proportional to the FOV when the atmosphere is transparent. As illustrated in Fig. 8, to take advantage of the smaller optical blur in the MWIR, must be less than 10. Maximum ranges approximately occurs at in the MWIR and at in the LWIR. These values correspond to detector sizes of 2 μm in the MWIR and 5 μm in the LWIR with optics.
Note that range is proportional to FOM () and FOM is a function of (). The aperture diameter appears in both quantities. As such, Figs. 6 to 8 are appropriate for fixed aperture analysis where the focal length is varied. If the focal length is held constant with the aperture diameter as a variable, the curves look quite different. Those graphs are provided in Sec. 7.
As illustrated in Fig. 9, straight lines4 originating at the origin represent constant range. Table 1 provides the required -number for for various detector sizes. Each LWIR combination, with optics or with optics, provides the same range. Likewise, each MWIR combination provides the same range. In the detector-limited region, the MWIR and LWIR ranges are equal. In the optics-limited region, the MWIR range is greater by the ratio of wavelengths under high signal-to-noise ratio conditions.
Required F-number for Fλ/d=2. Real optics usually has F>1.
The NEDT is a widely used measure of thermal imaging system sensitivity. For photon detectors, it can be calculated from first principles.8 It is routinely measured.9 Its value depends upon a myriad of parameters. Here, two different charge well capacities are considered, 6 million and 12 million electrons. These capacities are possible though careful design by expanding the well in the -direction. The integration time, , is variable and arbitrarily selected to maintain 50 percent well capacity, however, other values such 35 percent could have been used. The maximum integration time is limited by the frame rate. Assuming 60 Hz operation and allowing for readout time, the maximum integration time is set at 12 ms. The spectral response of the MWIR detector is 3.6 to 5.1 μm. The notch cold filter transmission at 4.2 μm is included in . The response of the LWIR detector is spectrally flat from 8 to 12 μm. For both, the quantum efficiency, , is 0.8 and the optical transmission, , is 0.8 with no spectral features.
The number8 of photoelectrons generated by the background at temperature is8 by the thermal derivative multiplied by the target . The signal-to-noise ratio is
The NEDT is independent of if the integration time changes to maintain a fixed well fill. This occurs when . For a particular waveband, the NEDT is constant with . Referring to Fig. 9, lines originating from the origin also represent constant NEDT for any specific background temperature.
Figure 10 provides the integration time as a function of background temperature for a typical MWIR system with 17 μm detectors. When the temperature decreases to for the well or for the well, the maximum integration time is reached. The NEDT is plotted in Fig. 11. The larger well permits a longer integration time and therefore, a lower NEDT. The difference between the wells is a factor of 2. Therefore, at any specific background temperature, the difference in integrations times is a factor of 2 and the difference in NEDT is .
The scarcity of photons in the MWIR makes this spectral region less desirable for low background temperature operation. As illustrated in Fig. 12 and by Eq. (13), the NEDT is inversely proportional to when and are fixed. This competes with the desire to have small detectors.
Figure 13 illustrates MWIR NEDT as a function of . The NEDT is constant when . Once the maximum integration time is reached, the NEDT increases. Most current detector sizes are greater than 17 μm and is typically less than 0.5. Thus, the NEDT variation with has been less objectionable so far. There are an abundance of photons in the LWIR and is always less than 12 ms. Figure 14 illustrated the LWIR variation in NEDT with . The seemingly high LWIR NEDT values are caused by the extremely short integration times of tens of microseconds to a few milliseconds depending upon and the background temperature.
There are two different system design approaches. The first is fixed array format and the second is fixed chip size. With fixed format, as the detector size decreases, the chip size decreases. The focal length must decrease to maintain the FOV. With fixed chip size, as the detector size decreases, the number of detectors increases and the FOV remains constant.
Fixed Array Format
Generally, search and detection are performed in the WFOV with identification performed in the NFOV. WFOV for tactical systems and, especially, in ground combat systems ranges from 10 deg to 40 deg. Changing the FOV is achieved by inserting a lens in the optical train or employing zoom optics. Using the small angle approximation, the ratio of WFOV to NFOV is simply the ratio of focal lengths. Practically, the FOV ratio should be about 3 to 6. If the ratio is greater, additional optics should be considered to create a medium FOV or a series of interim FOVs.
Aperture size is constrained by the platform. Typical aperture sizes on fighter aircraft, helicopters, and tanks are 6 inches for long range acquisition. Typical aperture sizes on small arms, dismount sights, and small scout systems are about 3 inches for short range acquisition. The NFOV will always use the large aperture. The WFOV requires a shorter focal length. Ideally, the -numbers should remain constant when switching FOVs to maintain the same contrast. This results in a smaller WFOV aperture.
Another choice is different -numbers for WFOV and NFOV. This is known as a dual -number system. For maximum search and detection range the -number should approach to maximize the signal as indicated by Eq. (11). For maximum identification ranges, should approach 2 as illustrated in Fig. 7. Dual or variable -numbers provide a performance enhancement10 over a dual FOV, single -number system. A dual FOV system is mechanically less robust due to moving lenses. A variable, dual -number, cold shield in the cryogenic space presents an even more challenging design.10
Assume the target contrast and target size is the same in both bands, atmospheric transmission is unity, and no noise is present. Using Eq. (10), the range is proportional to ()FOM. NVThermIP documentation7 recommends that the task difficulty parameter, , be 2.7 for detection and 18.8 for identification. In the detector-limited region, the FOM is proportional to and the range is inversely proportional to the FOV as illustrated in Fig. 15. For fixed FOVs, the detection range is or approximately 7 times greater than the identification range. This is offset by having the WFOV approximately 7 times the NFOV size. Small FOVs generally operate in the optics-limited region as illustrated in Fig. 16. In the optics-limited region, and range is proportional to 10.3().
Fixed Chip Size
A more interesting approach considers a fixed chip size. Representative sizes are 25 mm for long range acquisition systems and 12.5 mm for short range acquisition systems. A 2 μm MWIR detector will have pixels and a 5 μm LWIR detector will have pixels on a 25 mm chip. The detector count is one-half that for the 12.5 mm array.
The full focal plane would be mapped onto the display for the WFOV. In the NFOV, the central 512 pixels would be displayed. This FOV switch is performed entirely in software and, in fact, continuous zoom can be implemented electronically. Moving optics is not required to change FOV and dual -number cold shielding is not necessary. The WFOV to NFOV ratio is simply .
Practical Considerations (MWIR or LWIR)
The best wavelength band for any given task at hand depends upon the atmospheric transmission, atmospheric turbulence, background temperature, scene conditions, and target contrast. Some situations suggest a MWIR solution and others suggest a LWIR solution. It is impossible to satisfy all requirements. Various scenarios are briefly listed below.
In a humid environment, the atmospheric transmission is higher in the MWIR. As a result, many maritime and naval application sensors are MWIR. A cold, dry environment has a higher transmission in the LWIR. Systems designed by Nordic countries usually operate in the LWIR.
In the detector-limited region, the range is independent of wavelength. This means that MWIR and LWIR systems should provide the same range if the NEDTs are equal. However, when using NVThermIP, MWIR, and LWIR provide different ranges because the atmospheric conditions are different. Here, the atmospheric conditions are the primary performance discriminator and not the basic camera design.
Imaging through turbulence degrades11 MWIR performance more than LWIR range performance. For ground-to-ground applications, turbulence effects are larger than for air-to-ground and air-to-air applications.
As illustrated in Fig. 12, MWIR suffers significant performance degradation12 in cold environments. There is an abundance of photons in the LWIR so that the NEDT does not increase at low temperatures. At very cold temperatures, the identification range provided by a MWIR imager can be significantly less than that of a LWIR imager.
Historically, target acquisition systems for ground applications were LWIR to view through dirty battlefield conditions. Hot targets such as burning barrels, burning tanks, fires, and exhausts cause detector saturation and blooming. They also create veiling glare in the optical system. Due to a shift in Planck’s blackbody curve towards shorter wavelengths, hot target effects are an order of magnitude worse in MWIR than in LWIR. Under these conditions, a MWIR sensor can be rendered useless13 while a LWIR sensor can still be used to view the battlefield.
Combat obscurant smokes are somewhat transparent in the infrared. Experiments and analysis show that smoke14 does not degrade LWIR performance and degrades MWIR performance in a small to moderate, but not devastating, way. Smoke does not eliminate the MWIR long range identification advantage. Overall, there are a number of circumstances where ground forces, and to some extent aviation, rely on LWIR in dirty battlefield conditions where it is too risky to depend on a MWIR-only sensor.
Sensor motion, which occurs during panning and pilotage, increases image blur. Field studies15 have shown that image quality degrades when the integration time is greater than 4 ms when there is significant sensor motion. In the LWIR, the integration time is less than 4 ms for all background temperatures, to 50°C. This is not true in the MWIR where there is a scarcity of photons. As illustrated in Fig. 17, this small integration time only exists for small ratios. This suggests MWIR should not be used for pilotage or panning if .
The range predictions illustrated in Fig. 7 do not include atmospheric transmission or the NEDT. Section 6 listed factors that affect the NEDT. Using NVThermIP, ranges were predicted for a 0.79-inch fixed aperture MWIR sensor. The array contains 17 μm detectors. The integration time and performance parameter,8 , were varied to obtain the NEDT in Fig. 13 at the NVThermIP default background temperature of 27°C. The FOV and focal length were varied to obtain various values. Since the graph is a function of , choosing a larger aperture, such as 6 inches, affects the focal length and FOV, but the range remains constant. The other variables were target , , and . The atmospheric transmission was approximated by Beer’s law to avoid spectral transmission effects. Transmission per km, , was set at 0.7 to approximate poor weather and 0.85 for average weather conditions. The ranges are illustrated in Fig. 18. Equation (10) predictions were compared to the transparent atmosphere, , case. The difference, less than 8 percent, is believed due to system magnification16 issues. The predicted range with a real atmosphere, such as mid-latitude summer or subartic winter, can be quite different due to spectral transmission variations in the various environments.
Dual Band () Imager
The dual-band imager configuration optimizes many of the scenarios listed in Sec. 7.3 and also takes advantage of the increased MWIR range for high ratios as illustrated in Fig. 8. Generally, the LWIR is used for search and detection and the MWIR for identification. These ranges are illustrated in Figs. 15 and 19. It could also be a dual -number system. These bands can be used separately in a manner similar to two independent sensors that are used one at a time depending on the conditions and task that is favorable for a particular band. During adverse conditions such as cold weather, panning, and the dirty battlefield, the LWIR system is used exclusively. The LWIR ranges are illustrated in Figs. 15 and 16.
Comparing Figs. 16 and 19, MWIR range performance is 2.5 times that of a LWIR system in the optics-limited region. This is simply the ratio of wavelengths, . When adding atmospheric transmission to the calculation, the range performance is nearly double17 that of LWIR system with an equivalent aperture. In the detector-limit region, the ranges are the same.
The strong noise dependence on -number forces uncooled devices3 to be designed with optics. Uncooled detectors usually operate in the LWIR region. Therefore, to obtain with , the detectors must be 5 μm.
Based upon MTF theory, supported by sampling theory, and TTP range predictions, the desired detector sizes are 2 μm in the MWIR and 5 μm in the LWIR when coupled with optics when the signal-to-noise ratio is high.
There is a scarcity of photons in the MWIR. As a result, the NEDT increases dramatically as the ambient temperature decreases. Although MTF theory suggests a good design point is , larger detectors should be used in the MWIR to minimize the increased cold weather NEDT. As illustrated in Fig. 12, a detector size of 2.5 μm gives a reasonable NETD of 43 mK with optics at a background temperature of 10°C. Now, . Below 10°C, detector outputs could be binned or grouped to reduce the NEDT. Binning creates a larger effective detector size, reduces , and this reduces range at cold temperatures. This suggests that LWIR is the band of choice in a cold weather environment.
The selection of 2.5 and 5 μm detectors were based upon optics. It is much easier to design and fabricate an system. This increases the minimum detector size to 3 μm in the MWIR and 6 μm in the LWIR. Various combinations are provided in Table 1.
What does this mean for dual band focal planes? Ideally, one would desire a 2.5 μm (or 3 μm) MWIR detector and a 5 μm (or 6 μm) LWIR detector on the same focal plane. A conceptual architecture would provide four MWIR detectors for every larger LWIR detector in a sandwich configuration.
The approach is applicable to all imaging systems. It is a good metric that describes performance of all systems ranging from detector-limited to optics-limited operation. It is a good approach when the signal-to-noise ratio is high. This occurs with high atmospheric transmission, good target contrast, and good background temperatures resulting in good apparent target contrast. With optics, the smallest useful detector size is 2 μm in the MWIR and 5 μm in the LWIR. With challenging signal-to-noise conditions and more realistic optics, the smallest useful detector size is 3 μm in the MWIR and 6 μm in the LWIR.
In the detector-limited region, range is inversely proportional to the FOV. In the optics-limited region, range is proportional to the aperture diameter.
An exciting approach is to fabricate a large chip with many detectors. Now variable FOVs are possible with the same optics. This provides a single system optimized both for target detection and target identification. The constant chip size and smaller detector approach provides a number of advantages over current system designs. It provides long range diffraction-limited target identification at low -number commensurate with past system of larger apertures. It provides instant NFOV to WFOV switching or continuous zoom controlled by software with no moving optics. It also provides a physically smaller imager than current high -number systems.
There are many benefits to approaching these detector sizes with low -number optics to include less expensive detectors, no need for dual FOV or continuous zoom optics, and no need for dual -number optics. Our approach provides the smallest volume and lowest weight imager with maximum range performance. While this paper focused on infrared design, the approach applies to all imaging sensors.
Gerald C. Holst is president of JCD Publishing Company and a consultant for imaging system analysis and testing. He has chaired the SPIE conference Infrared Imaging Systems: Design, Analysis, Modeling and Testing since 1989. He is author of over 30 journal articles and 6 books (published by SPIE and/or JCD Publishing). He is a member of OSA and IEEE and is a SPIE Fellow.
Ronald G. Driggers was appointed to the Senior Executive Service as the superintendent of the Optical Sciences Division at the Naval Research Laboratory in 2008. He currently manages the efforts of more than 200 scientists and engineers and over $100M in research and development programs. Previously, he was the director of the Modeling and Simulation Division at the U.S. Army’s Night Vision and Electronic Sensors Directorate (NVESD).