This PDF file contains the editorial “Why Don’t Optical Engineers Publish?” for OE Vol. 38 Issue 05

This PDF file contains the editorial “Guest Editorial: Special Section on Sampled Imaging Systems” for OE Vol. 38 Issue 05

By rigorously extending modern communication theory to the assessment of sampled imaging systems, we develop the formulations that are required to optimize the performance of these systems within the critical constraints of image gathering, data transmission, and image display. The goal of this optimization is to produce images with the best possible visual quality for the wide range of statistical properties of the radiance field of natural scenes that one normally encounters. Extensive computational results are presented to assess the performance of sampled imaging systems in terms of information rate, theoretical minimum data rate, and fidelity. Comparisons of this assessment with perceptual and measurable performance demonstrate that (1) the information rate that a sampled imaging system conveys from the captured radiance field to the observer is closely correlated with the fidelity, sharpness and clarity with which the observed images can be restored and (2) the associated theoretical minimum data rate is closely correlated with the lowest data rate with which the acquired signal can be encoded for efficient transmission.

Two perception experiments are conducted to quantify the relationship between imager sampling artifacts and target recognition and identification performance using that imager. The results of these experiments show that in-band aliasing (aliasing that overlaps the baseband signal) does not degrade target identification performance, but out- of-band aliasing (such as visible display raster) degrades identification performance significantly. Aliasing had less impact on the recognition task than the identification task, but both in-band and out-of-band aliasing moderately degrades recognition performance. Based on these experiments and other results reported in the literature, it appears that in-band aliasing has a strong effect on low-level discrimination tasks such as point (hot-spot) detection; out-of-band aliasing has only a minor impact on these tasks. For high-level discrimination tasks such as target identification, however, out-of-band aliasing has a significant impact on performance, whereas in-band aliasing has a minor affect. For intermediate-level discrimination tasks such as target recognition, both in-band and out-of-band aliasing have a moderate impact on performance. Based on data from the perception experiments, the modulation transferfunction (MTF) squeeze model is developed. The degraded performance due to undersampling is modeled as an effective increase in system blur or, equivalently, a contraction or "squeeze" in the MTF. An equation is developed that quantifies the amount of MTF squeeze or contraction to apply to the system MTF to account for the performance degradation caused by sampling.

The minimum temperature difference perceived (MTDP) is a new device figure of merit for assessing undersampled thermal imagers. MTDP is based on the perception of the standard four-bar test pattern, like minimum resolvable temperature difference (MRTD). In undersampled imagers, the test pattern image is increasingly distorted toward higher pattern frequencies due to aliasing. Contrary to MRTD, MTDP takes into account such signal distortions to a certain extent. With MTDP the imager performance can be assessed beyond the Nyquist frequency. The modulation in the distorted four-bar pattern images is studied as a function of pattern frequency and position. We show that for each pattern frequency, an optimum phase position can be found at which the average modulation in the reproduced pattern image has a maximum. This modulation is defined as the average modulation at optimum phase (AMOP). The MTDP is the temperature difference at which four, three or two bars can be resolved by an observer, with the test pattern at the optimum phase position. The MTDP approach is implemented in the TRM3 model. Examples comparing measured and predicted MTDP data for several staring imagers are presented. The validity of MTDP to predict the range performance of undersampled imagers is examined and results are presented.

The image quality resulting from a 2-D image-sampling process by an array of pixels is described. The description is based on a Fourier transformation of the Wigner-seitz cell, which transforms a unit cell of the sampling lattice in the spatial domain into a bandwidth cell in the spatial-frequency domain. The area of the resulting bandwidth cell is a quantitative measure of the image fidelity of the sampling process. We compare the image-quality benefits of three different oversampling geometries in terms of the modulation transfer function (MTF) as a function of the amount of oversampling used.

Many modeling, simulation and performance analysis studies of sampled imaging systems are inherently incomplete because they are conditioned on a discrete-input, discrete-output model that only accounts for blurring during image acquisition and additive noise. For those sampled imaging systems where the effects of digital image acquisition, digital filtering and reconstruction are significant, the modeling, simulation and performance analysis should be based on a more comprehensive continuous-input, discrete-processing, continuous-output end-to- end model. This more comprehensive model should properly account for the low-pass filtering effects of image acquisition prior to sampling, the potentially important noiselike effects of the aliasing caused by sampling, additive noise due to device electronics and quantization, the generally high-boost filtering effects of digital processing, and the low-pass filtering effects of image reconstruction. This model should not, however, be so complex as to preclude significant mathematical analysis, particularly the mean-square (fidelity) type of analysis so common in linear system theory. We demonstrate that, although the mathematics of such a model is more complex, the increase in complexity is not so great as to prevent a complete fidelity-metric analysis at both the component level and at the end-to-end system level; that is, computable mean-square-based fidelity metrics are developed by which both component-level and system-level performance can be quantified. In addition, we demonstrate that system performance can be assessed qualitatively by visualizing the output image as the sum of three component images, each of which relates to a corresponding fidelity metric. The cascaded, or filtered, component accounts for the end-to-end system filtering of image acquisition, digital processing, and image reconstruction; the random noise component accounts for additive random noise, modulated by digital processing and image reconstruction filtering; and the aliased noise component accounts for the frequency folding effect of sampling, modulated by digital processing and image reconstruction filtering.

A general method is outlined to increase the spatial resolution of digital video sequences containing uncontrolled camera motion. This is accomplished by combining multiple frames of video into a composite image of a higher spatial sampling rate than the original video. Fundamental to this technique is the computation of optical flow. This technique has the greatest impact on camera systems where the focal plane array spatially undersamples the projected optical image. Implicit to the technique is the potential trade-off between frame rate, latency, and display resolution in a digital imaging system.

Many imaging systems utilize detector arrays that do not sample the scene according to the Nyquist criterion. As a result, the higher spatial frequencies admitted by the optics are aliased. This creates undesirable artifacts in the imagery. Furthermore, the blurring effects of the optics and the finite detector size also degrade the image quality. Several approaches for increasing the sampling rate of imaging systems have been suggested in the literature. We propose an algorithm for resolution enhancement that exploits object motion in digital video sequences. Unlike previously defined techniques, we use an automated segmentation method to isolate rigid moving objects. These are accurately registered and the multiple observations of the object are used to produce an effectively high sampling rate over the object. The experimental results presented illustrate the breakdown of resolution enhancement algorithms that assume global scene motion when the actual scene motion is nonglobal. The performance of the proposed algorithm is illustrated using images from a forward looking IR imager and a visible range camera.

Commercial remote sensing systems generally employ a linear detector array for imaging the ground scene. Strategies for increasing the ground sampling along one direction of the array can be employed to improve the image quality. Image simulations were generated to quantify the image quality improvement, in terms of the National Image Interpretability Scale (NiIRS), as the sampling is increased in the along-scan (A/S) direction of the array. The simulations modeled a remote sensing system with ?FN/p = 1 (where FN is the system f-number and p is the detector sampling pitch) and show that an image quality improvement of approximately 0.35 NIIRS can be achieved if the sampling rate is increased in the A/S direction by 2x

Space-based high-resolution scanning array imaging systems have the potential to introduce large amounts of image smear. When designing these systems, it is useful to understand how smear will degrade image quality. A brief description of the causes of smear and a simple mathematical model are presented. A series of image simulations (for a system in which ? FN/p = 1.0, where ? is the mean wavelength for a panchromatic system, FN is the system f number, and p is the pixel pitch of the detectors) are performed in which along scan smear (ranging from 1.0 to 8.0 pixels) is introduced. Using the National Imagery Inter- pretability Rating Scale (NIIRS), expert observers rated ?NIIRS difference in image quality between the images with simulated smear and the original "unsmeared" image. The functional relationship between smear error and image quality (in units of?NIIRS) is determined.

Electro-optic sensor simulation and sensor design require a common understanding of spatial sampling. As part of the development of a high fidelity sensor simulation, the U.S. Army Night Vision Electronic Sensors Directorate has developed a methodology that enables the simulation designer to minimize the spatial sampling rate of the input image based on the sensor parameters. Gabor information theory is used to define the limit imposed on the ability to simultaneously represent a spatial function and its Fourier transform. This is then coupled with the necessary consequences of sampling the image to develop limits that ensure minimal error. Resolution requirements for synthetic presensor imagery can be developed using this formalism. These requirements are not based on the instantaneous field of view (IFOV) or on the detector width alone but are based on the full sensor point spread function.

. A technique for determining relative degradations from image metrics is presented along with a technique for predicting sensor performance from metrics. These techniques are illustrated with degradations of blur and noise in thermal imagery. These uses of metrics are depicted as mappings among a degradation space, an image quality metric space, and an operational performance space. This technique has utility in sampled imagery applications where input and output image comparison is possible, e.g., validation of an infrared scene projector (IRSP), testing image compression algorithms, image simulation, etc. Such applications have a known input image and a degraded output image. With the input image, one can characterize the output image in terms of its degradations relative to the input. The concept of a degradation space leads to developing an Operational Performance Metric (OPM) in terms of more traditional Image Quality Metrics (IQMs). The technique is illustrated using empirical results for human observers performing recognition tasks with thermal imagery in a degradation space of blur and noise.

Experimental results of dynamic minimum resolvable temperature (DMRT) testing of a staring array thermal imager are reported. Temporal effects of scanning four bar targets are discussed. The impact of target phase for static minimum resolvable temperature (MRT) tests is presented and shown through measured results. Details of the DMRT test are offered as guidelines for such tests. DMRT test results are shown as a function of scanning speed. Results for good image uniformity are compared to results with poor image uniformity. The experimental results are used to determine the perceived resolution increase as a function of target speed. An optimum scanning speed is identified and discussed. The DMRT results at this optimum scanning speed are compared to the static MRT results.

This paper is a tutorial on the laboratory measurement of sampled infrared imager performance. A review of the classical infrared imager performance measurements that describe sensitivity, resolution, and human interpretation performance is presented. These parameters are Noise Equivalent Temperature Difference (NETD), Modulation Transfer Function (MTF), and Minimum Resolvable Temperature Difference (MRTD or just MRT), respectively. The sensitivity parameters are extended to undersampled imaging systems with 3-D Noise, Inhomogeneity Equivalent Temperature Difference (IETD), and correctability. Also, the measurement of the detector MTF of sampled systems is described using either a scanning slit or tilted edge. Finally, the MRT of sampled systems is a strong function of target to sensor phase. A solution to the phase dependence and sampling limitations is the dynamic MRT, or DMRT. All of these tests are presented in detail and the problems associated with the laboratory measurement of sampled imaging systems are described.

The temperature resolution of infrared focal plane arrays is limited by temporal and spatial noise. The spatial noise usually is partially removed by correction procedures. These correction procedures reduce the spatial noise to a magnitude below the temporal noise. The correctability c defined as the ratio of the spatial to the temporal noise is a figure of merit for the state of the correction. We consider the transient degradation of the correctability after correction. A new figure of merit, the long-term stability time constant ?/ts is introduced. This time indicates the duration after a nonuniformity correction during which the spatial noise increases to values higher than that of the temporal noise. Several staring infrared focal plane arrays differing in size and in detector material are investigated. The correctability c is determined after various correction procedures and the long-term stability time ?/ts is measured. The degradation of the correctability is caused by a few individual pixels in the detector array. We can classify three different types of ''bad pixels," which degrade the correctability. These are weak pixels that show a low responsivity and flickering and drifting pixels that show excessive 1/f-noise.

We present the design and application of subwavelength diffractive lenses for integration with IR photodetectors. In addition to 100% fill factor, low f /numbers, and the ability for monolithic integration, subwavelength lenses offer high diffraction efficiencies yet require only single step fabrication. However, their design is made difficult due to the subwavelength nature, which precludes the use of scalar diffraction theory, and finite extent, which prevents the use of rigorous coupled wave techniques. To overcome these limitations we employ a design algorithm that uses the boundary element method as a vector model of diffraction. We apply our algorithm to the design of subwavelength lenses for integration with a microbolometer and a quantum well IR photodetector.

A general approach for eliminating higher order aliasings is developed by using multiple interlaced samplings. The M-fold interlaced samplings can be used to eliminate aliasing up to the M'th order by solving a set of simultaneous linear equations involving an MxM Vandermonde matrix, thereby increasing the sampling rate by M times. The concept is demonstrated by considering the case of Fresnel zone sampling and reconstruction.

Joint transform correlation involves the recording and display of the joint power spectrum by sampling devices. The Nyquist condition for sampling a joint power spectrum depends on the distance between the input subimages, the size of the subimages, the focal length of the Fourier lens, and the light wavelength. Since the joint power spectrum is equivalent to a bandpass function, an undersampled joint power spectrum will lead to multiple interlaced outputs and the correlation spots and DC terms could either overlap or remain separate. The specifications pertaining to the sampling device are identified so as to be able to distinguish the correlation spots of interest in the interlaced output.

Rational filters are extended to multichannel signal processing and applied to image interpolation. Two commonly used decimation schemes are considered: a rectangular grid and a quincunx grid. For each decimation lattice, we propose a number of adaptive resampling algorithms based on the vector rational filter (VRF). These algorithms exhibit desirable properties such as edge and detail preservation and accurate chromaticity estimation. In these approaches, color image pixels are considered as three-component vectors in the color space. Therefore, the inherent correlation that exists between the different color components is not ignored. This leads to better image quality compared to that obtained by componentwise or marginal processing. Extensive simulations show that multichannel image processing with the proposed algorithms (VRFL) and (VRFd) based on lp-norm and directional processing, respectively; significantly outperform linear and some nonlinear techniques, e.g., vector FIR median hybrid filters (VFMH). Some images interpolated using VRFL and VRFd are presented for qualitative comparison. These images are free from blockiness and jaggedness, confirming the quantitative results.

The fixed-polarizer ellipsometer measures thickness of thin films. It is simple, inexpensive, and provides a linear response over a range of 800 Å. We develop a matrix formulation to describe the optical characteristics of the instrument and apply it to the case of a single thin film on a substrate. Excellent agreement is found between experimental and simulated results. Applying the instrument to optical immunoassay, we show that its sensitivity can extend to 4 pg/ml, depending upon the analyte. This compares favorably with commercially available manual and automated immunoassay systems. The fixed-polarizer ellipsometer appears to be well-suited for use in laboratory and production environments.

An Encoder-Segmented Neural Network (ESNN)-based approach is proposed to improve the efficiency of image segmentation. The features are ranked according to the encoder indicators by which the insignificant features will be eliminated from the original feature vectors and the important features reorganized as the encoded feature vectors for the subsequent clustering. The ESNN developed can improve on the existing Fuzzy c-Means (FCM) algorithm in feature extraction. The cluster number selection can be accomplished automatically. This method was successfully implemented for automatic labeling of tissues in MR brain images. Experimental results show that the ESNN-based approach offers satisfactory performance in both efficiency and adaptability.

A rosette scanning infrared seeker (RSIS) is a single-detector scanner with a rosette scan pattern, and offers the positioning and imaging information about a target to the servo system of the missile. An instantaneous field of view (IFOV) of the RSIS is a diameter of the infrared (IR) detector. To be insensitive to the interference of background signals and detector noise, the IFOV should be small. If the IFOV is designed too small, however, it may not achieve full scan coverage (FSC). The invisible regions cause the performance of the RSIS to deteriorate. We present an analysis of scan coverage of the IFOV and an efficient IR counter-countermeasure (IRCCM) using the moment technique with two threshold levels. To evaluate the tracking performance of the RSIS dynamically, we simulate it in various situations. The simulation results show that the RSIS using our IRCCM can eliminate the effects of countermeasures (CM) such as IR flares.

We report a simple technique for determining the orientation of the extraordinary axis at the input side of a twisted-nematic liquid crystal spatial light modulator. In this approach, we illuminate the display with linearly polarized light and examine the diffraction pattern as we change the external voltage applied to the display. When the incident light is polarized along the extraordinary axis, we see changes in the diffraction pattern as a function of applied voltage. Experimental measurements and theory are presented.