At DSTO, a real-time scene generation framework, VIRSuite, has been developed in recent years, within which trials data are predominantly used for modelling the radiometric properties of the simulated objects. Since in many cases the data are insufficient, a physics-based simulator capable of predicting the infrared signatures of objects and their backgrounds has been developed as a new VIRSuite module. It includes transient heat conduction within the materials, and boundary conditions that take into account the heat fluxes due to solar radiation, wind convection and radiative transfer. In this paper, an overview is presented, covering both the steady-state and transient performance.
Continuing interest exists in the development of cost-effective synthetic environments for testing Laser Detection and
Ranging (ladar) sensors. In this paper we describe a PC-based system for real-time ladar scene simulation of ships and
small boats in a dynamic maritime environment. In particular, we describe the techniques employed to generate range
imagery accompanied by passive radiance imagery. Our ladar scene generation system is an evolutionary extension of
the VIRSuite infrared scene simulation program and includes all previous features such as ocean wave simulation, the
physically-realistic representation of boat and ship dynamics, wake generation and simulation of whitecaps, spray,
wake trails and foam. A terrain simulation extension is also under development. In this paper we outline the
development, capabilities and limitations of the VIRSuite extensions.
We describe recent improvements in our maritime scene generation program and the extension in capabilities that has
been achieved. The motion of multiple boats under independent control can now be simulated, as well as large ship
motion. The effects we simulate include ocean surfaces in different sea states, the physically-realistic representation
of boat and ship dynamics, wake generation and generation of surface effects including whitecaps, spray, wake trails
and foam. We describe our graphical user interface tools, the underlying phenomena that they control and their
application in enabling versatile real-time maritime scene simulation.
With the development and widespread availability of computer graphics cards, complex infrared scenes can now be
readily generated for application in real-time hardware-in-the-loop simulations. It is important that the best efforts are
made to ensure that the scenes are radiometrically valid, to the level where the operation of the imaging infrared unitunder-
test can be properly emulated. In this paper we describe the techniques we employ to ensure radiometric
validity within our real-time aircraft and boat simulation applications of current interest.
We describe the extension of our real-time scene generation software VIRSuite to include the dynamic simulation of
small boats and their wakes within an ocean environment. Extensive use has been made of the programmabilty
available in the current generation of GPUs. We have demonstrated that real-time simulation is feasible, even
including such complexities as dynamical calculation of the boat motion, wake generation and calculation of an FFTgenerated
We assess the issues that need to be addressed to ensure that a resistor array infrared projector is capable of validly
simulating the real world. These include control of the additional sources of blurring and aliasing arising from the
presence of the projector and its associated scene generation system, nonuniformity correction, busbar robbing,
spurious back reflections and narcissus. In particular, we reconfirm that a 2 × 2 projector/unit-under-test pixel
mapping ratio offers a good compromise for controlling the additional blurring and aliasing, and furthermore, we
demonstrate achievement of projector nonuniformity noise equivalent temperature differences (NETDs) in the 20 mK
Resistor array infrared projector nonuniformity correction (NUC) is currently limited in fidelity. In the flood
technique a fundamental limitation has been the inevitable presence of Moire fringes. In this paper, an advanced NUC
procedure is described in which the Moire patterns are successfully subtracted, leading to improved levels of residual
nonuniformity. It is shown that, irrespective of the projection technology, the Moire fringes exist at the unit-under-test
image plane where they appear in general as sampling noise. Their control through choice of mapping ratio is
Research leading towards the continued improvement in resistor array infrared projector nonuniformity correction
(NUC) is reported, particularly at low drive levels relevant to thermal imager and FLIR test and evaluation
applications. Moire fringes have been successfully compensated, as has the checkerboard effect seen in earlier flood
NUC measurements. With these improvements, the residual nonuniformity associated with the random spatial noise
has been reduced successfully to the 0.1-0.2% rms level, equivalent to 20-60 mK noise equivalent temperature
differences. The random noise is accompanied, however, by a low spatial frequency fixed pattern, currently
unexplained but possibly attributable to busbar robbing in the electronic backplane.
VIRSuite, the GPU-based suite of software tools developed at DSTO for real-time infrared scene generation, is
described. The tools include the painting of scene objects with radiometrically-associated colours, translucent object
generation, polar plot validation and versatile scene generation. Special features include radiometric scaling within the
GPU and the presence of zoom anti-aliasing at the core of VIRSuite. Extension of the zoom anti-aliasing construct to cover target embedding and the treatment of translucent objects is described.
Results from sparse grid and flood nonuniformity correction (NUC) obtained using the DSTO Primary Infrared Scene
Projection at 1:1 mapping ratio are reported. Residual nonuniformities in the 0.5-1.0% range are currently being
achieved, the flood results equating to noise equivalent temperature differences in the 50-100mK range within the low
drive thermal imager and FLIR simulation region. The NUC techniques and results are discussed in the light of both
their present applicability and scope for further improvement.
The zoom anti-aliasing (ZAA) procedure used for rendering computer-generated targets at long range is examined in the light of its lack of conformality with sampling theory. This has led to the development of a GPU-based conformal version, called Schade ZAA. It is shown that Schade ZAA leads to improvement in intensity errors and significantly less scintillation when the target subtends less than a single pixel in screen space. It is shown further that the intensity errors for the normal type of irregularly-shaped target of most user interest are considerably smaller than has been previously reported. This is a significant result in that it gives confidence that the intensity of small and point source targets can be successfully conserved within the ZAA process.
Resistor array infrared projectors offer the unique potential of simultaneously covering both a wide apparent temperature range and providing fine temperature resolution at low output levels. The temperature resolution capability may not be realized, however, if the projector error sources are not controlled; for example, residual nonuniformity after nonuniformity correction (NUC) procedures have been applied, temporal noise in analog drive voltages and quantization at several points in the projection system, all of which may introduce errors larger than the desired resolution. In this paper the temperature resolution limits are assessed in general. In particular, the quantization errors are assessed and the post-NUC residual nonuniformity levels required for achievement of fine temperature resolution are calculated.
Numerous infrared scene projection technologies have been investigated since the 1970s. Notably, from the late 1980s the development of the first resistor array infrared projectors gained leverage from the strong concurrent developments within focal plane array imaging technology, linked by the common need for large integrated circuits comprising a 2-dimensional array of interconnected unit cells. In the resistor array case, it is the unit cell comprising the resistively heated emitter and its dedicated drive circuit that determines the projector response to its associated scene generator commands. In this paper we review the development of resistor array technology from a historical perspective, concentrating on the unit cell developments. We commence by describing the technological innovations that forged the way, sharing along the way stories of the successes and failures, all of which contributed to the steady if somewhat eventful growth of the critical knowledge base that underpins the strength of today's array technology. We conclude with comments on the characteristics and limitations of the technology and on the prospects for future array development.
Results from application of the sparse grid nonuniformity correction procedure within the DSTO resistor array Primary Infrared Scene Projection system are reported. In particular, the techniques used to cover the full dynamic range and to combat camera drift are described. The effectiveness of the projector NUC procedure is assessed and discussed in terms of the scope for further improvement.
Array nonuniformity is the dominant factor limiting the temperature resolution of the current generation of emissive dynamic infrared scene projectors. Over the past five years or so numerous papers have been presented associated with the measurement of the array nonuniformities and the design and implementation of efficient nonuniformity correction (NUC) techniques. A considerable amount of progress has been made towards achieving the desired NUC goals. A number of factors, however, limit the achievement of fine temperature resolution within emissive infrared projection systems, improvements still being needed to achieve residual nonuniformity levels low enough to satisfy the demanding requirements of low NETD thermal imaging systems. In particular, the NUC camera has a strong influence on the effectiveness of the projector NUC procedure. In this paper we describe an alternative method for collecting projector NUC data that relies on the use of several integration times and also multiple calibration points for correcting the camera nonuniformities, the method being designed to improve the accuracy of the projector NUC procedure.
Proc. SPIE. 5408, Technologies for Synthetic Environments: Hardware-in-the-Loop Testing IX
KEYWORDS: Nonuniformity corrections, Cameras, Resistance, Field programmable gate arrays, Control systems, Data processing, Projection systems, Infrared radiation, Field effect transistors, Algorithm development
The new generation PC-based array control electronics (PACE) system for emissive infrared projector real-time scene data processing has opened the potential for the development of more complex real-time nonuniformity correction (RNUC) algorithms than were formerly possible. In this paper, emitter array response data are analyzed in order to identify the underlying physical processes and to identify the form of the RNUC algorithm they suggest. It is shown that although the PACE system is capable of processing the algorithm, the development of a practical RNUC processor would seem to be limited by the complexities that underlie the observed variability in emitter response.
Aliasing is unavoidable in real-time computer image generation due to the sampling processes occurring within the graphics hardware. In particular, aliasing produces scintillation effects and significant radiometric inaccuracy when targets are rendered at long range. This problem was alleviated some years ago by the development of a zoom anti-aliasing (ZAA) technique within infrared missile seeker hardware-in-the-loop simulations. An alternative ZAA technique based on extensive use of available graphics hardware functions is described here and compared to the original technique.
A challenging aspect of real-time infrared scene generation for the hardware-in-the-loop testing of infrared-guided weapon systems is the rendering of particle systems to represent gaseous and particulate volumes. In this work, a simulation tool is described for enabling the generation of real-time particle effects with high spatial and temporal fidelity by using many less primitives than traditional means. The tool can be applied for representing plumes and countermeasures, and enables the simulation of several key capabilities, including internal flow, turbulence, persistence, and structure, all capable of being varied dynamically as a function of power setting at the source. The principles of operation, software implementation, and general performance are discussed.
A challenging aspect of real-time infrared scene generation for the hardware-in-the-loop (HWIL) testing of infrared-guided weapon systems is the rendering of particle systems to present gaseous and particulate volumes. In this paper a simplified technique is described for generating real-time particle effects with high spatial and temporal fidelity by using many less primitives than traditional means. The technique is suitable for representing plumes and countermeasures and enables the simulation of several key capabilities including internal flow, turbulence, persistence and structure, all capable of being varied dynamically as a function of power setting at source. The principles of operation, software implementation and general performance are discussed.
The DSTO Primary Infrared Scene Projection (PIRSP) system has been used to investigate the practical application of the emitter array flood nonuniformity correction (NUC) technique. In the first instance the measurements have been limited to the special case of unity mapping ratio. The methods for achieving unity mapping at sub-pixel registration are described; in particular, the use of Moire fringes for accurately measuring the optical distortion across the field-of-view and for attaining the optimal mapping condition. Application of the flood NUC technique within the PIRSP system is discussed in terms of its convergence limitations. The latter include the presence of spatial and temporal camera noise, optical distortion, the mixing of neighbouring pixel information due to the finite point spread function and radiance-to-voltage transformation errors.
Aliasing is unavoidable in real-time computer image generation due to the sampling processes occurring within the graphics hardware. In particular, aliasing produces scintillation effects and significant radiometric inaccuracy when targets are rendered at long range. The zoom anti-aliasing techniques designed to alleviate the inherent aliasing problems are reviewed here. It is shown that since these rely on computational power rather than on optimal use of the extensive set of functions available within the graphics hardware they tend to be slower and more complex than necessary. A new technique based on use of the graphics hardware functions is described and compared to the earlier techniques. It is shown that the technique is faster and less complex while being similarly capable in reducing the level of aliasing.
A new infrared projector emitter response curve-fitting procedure suitable for generating nonuniformity coefficients capable of being applied in existing real-time processing architectures is introduced. The procedure has been developed through detailed analysis of a Honeywell Multi-Spectral Scene Projector (MSSP) sparse array data set, combined with an appreciation of the underlying physical processes that lead to the generation of infrared radiance.
The search for optimal IR scene projection nonuniformity correction procedures reported in earlier papers is continued. In this paper the application of the flood nonuniformity correction procedure described earlier is extended to the case where only a sublattice of projector pixels is lit, enabling nonuniformity correction for the practically interesting case of greater-than-unity mapping ratios.
Proc. SPIE. 4717, Technologies for Synthetic Environments: Hardware-in-the-Loop Testing VII
KEYWORDS: Nonuniformity corrections, Error analysis, Computer simulations, Data processing, Signal processing, Black bodies, Projection systems, Infrared radiation, Field effect transistors, Temperature metrology
An alternative class of infrared projector real-time nonuniformity correction processor is introduced, based on the concept that the fundamental role of the processor is to reverse each of the projector processing steps as the input DAC voltage word is converted into infrared signal radiance output. The design is developed by assessment of the sequence of processes occurring within the projector and is tested by simulation. It is shown that there is potential for high fidelity nonuniformity correction across the infrared dynamic range without the need for the introduction of curve-fitting breakpoints.
The search for optimal infrared scene projection nonuniformity correction procedures reported in earlier papers is continued. in this paper the application of the specialized flood nonuniformity correction algorithm described earlier is extended to the more practical case where the pixel-to-pixel mapping is imperfect.
The thermal conduction and electronic drive processes that govern the temporal response of resistor array infrared projectors are reviewed. The characteristics and limitations of the voltage overdrive method that can be implemented for sharpening the temporal response are also discussed. Overdrive is shown to be a viable technique provided sufficient drive power and temperature margins are available outside of the normal dynamic range. It is shown also by analysis of overdrive measurements applied to a Honeywell GE snapshot resistor array that practical real-time overdrive processors can be designed to operate consistently with theoretical predictions.
Proc. SPIE. 4027, Technologies for Synthetic Environments: Hardware-in-the-Loop Testing V
KEYWORDS: Nonuniformity corrections, Thermography, Infrared imaging, Mid-IR, Transform theory, Projection systems, Infrared radiation, Commercial off the shelf technology, Temperature metrology, RGB color model
It is shown that commercial-off-the-shelf (COTS) renderers can be used for covering the simultaneous fine temperature resolution and large dynamic range specifications associated with the demands of medium-wave infrared scene projection applications. Appropriate use of the RGB capabilities of the COTS renderer combined with redistribution of the binary scene data by using a nonlinear transformation enables the dual specifications for 0.1 degree Celsius small signal temperature resolution and > 400 degree Celsius range in simulated temperature difference to be simultaneously met.
The one-dimensional nonuniformity scene generation method presented in an earlier paper is extended to the two- dimensional case of real interest. It is shown that the algorithm applied to the one-dimensional case is extendable although its speed of convergence is reduced in two dimensions because of the increased mixing of nonuniformity information. Alternative nonuniformity correction algorithms are developed and compared and it is demonstrated that by utilizing an estimate of the point spread function the scene correction efficiency can be substantially improved.
An analytical model is developed suitable for optimizing the choice of the design parameters that govern the extent of spatial filtering and artifact generation (aliasing) in dynamic infrared scene projection systems. The filtering and sampling processes occurring at both the scene renderer and infrared projector are examined quantitatively in such a way that they can be compared to the filtering and sampling processes occurring within the imaging infrared unit-under- test. Graphs and formulae are included that enable the optimal choice of the relevant system parameters and allow quantitative assessment of the additional sources of spatial filtering and aliasing present within the infrared projection system.
Image filtering in sampled dynamic infrared scene projection systems is examined from the point of view of providing an improved insight into the choice of the pixel mapping ratio between the projector and imaging unit-under-test. The 2D vector analysis underlying the transfer of image information in such systems is reviewed and is applied to the dynamic infrared scene projection case. It is shown that the 4:1 (2 X 2:1) pixel mapping ratio previously recommended in a desirable criterion from the spatial fidelity viewpoint, particularly when high spatial frequency information represented by point sources and scene edges is being projected. Cost constraints can, however, prevent the 4:1 mapping ratio from being met, in which case the effects on hardware-in-the-loop simulation validity need to be examined carefully. The vector analysis presented here provides a tool useful for the future examination of such cases.
The factors influencing the attainment of fine simulated temperature resolution in a dynamic infrared scene projection system are examined. It is shown that the input data word bit resolution, the simulated temperature range requirement, the shape of the input word transfer function and the presence of spatial noise all affect the temperature resolution. Expressions are derived for allowing the simulated temperature resolution of both emissive projectors and spatial light modulator projectors to be estimated. Projector spatial noise is also analyzed in order to allow the noise limit on temperature resolution to be determined in terms of array nonuniformity. The array uniformity goals to which infrared projector manufacturers need to strive after nonuniformity correction procedures have been applied are therefore determined.
The spatial and temporal noise factors that set the ultimate performance limits on dynamic infrared scene projectors are assessed. Fixed pattern noise and the needs for non-uniformity correction procedures are both discussed, followed by a detailed analysis of the temporal and residual spatial noise contributions within the projector/imaging unit-under-test (UUT) composite system. It is found that the system is more tolerant to projector spatial noise by a factor of about four compared to UUT spatial noise. The temporal and spatial noise contributions are combined appropriately in order to provide guidance as to the levels of array uniformity that will need to be achieved if the expected demands for fine temperature resolution are to be met.
Over the past five years there has been steady development in the field of dynamic infrared scene generation (DIRSP), to the point where the first generation of dynamic infrared scene projectors can now be subjected to critical analysis. In this paper, the specific areas of optical projection validity, spectral emissivity and temperature resolution are discussed in relation to the viability of the DIRSP systems to which they apply.
The importance of testing IR imagers and missile seekers with realistic IR scenes warrants a review of the current technologies used in dynamic infrared scene projection. These technologies include resistive arrays, deformable mirror arrays, mirror membrane devices, liquid crystal light valves, laser writers, laser diode arrays, and CRTs. Other methods include frustrated total internal reflection, thermoelectric devices, galvanic cells, Bly cells, and vanadium dioxide. A description of each technology is presented along with a discussion of their relative benefits and disadvantages. The current state of each methodology is also summarized. Finally, the methods are compared and contrasted in terms of their performance parameters.
In recent years there have been rapid developments in the field of dynamic infrared scene projection, driven principally by the demands for real-time hardware-in-the-loop simulation and by the fortuitously concurrent advances in large array silicon surface micromachining techniques and addressing/multiplexing scheme electronics. A diversity of techniques have been explored towards satisfying the infrared projection requirements, resulting in a need for development of systematic comparative assessment procedures. In this paper the current status of infrared projector analysis is reviewed.
The speed demands that determine the frame rate requirements for dynamic infrared scene projectors are discussed together with the speed characteristics and limitations of the scene projectors currently being developed. Multiplexing/addressing scheme limitations are discussed and specific infrared projection technologies are surveyed with particular attention given to the design compromises that tend to determine speed capability.
The effective blackbody temperature used to specify the radiant output from an IR projector is defined formally as the temperature assumed by the blackbody target that generates the same signal radiance as the IR projector characterized. Analytical expressions and graphical analyses are developed enabling the calculation of the blackbody temperature in terms of both the actual device temperature and a specified active zone attenuation coefficient, the latter quantifying the deviation from blackbody behavior. Both the projector signal radiance and the signal current developed in the imaging unit under test illuminated by the projector are shown to be readily calculable once the value of the effective blackbody temperature is known. The analysis is applied to the comparison of the thin film resistor, silicon bridge resistor, and suspended membrane resistor emissive IR projection technologies.
The optical considerations that need to be addressed during the design of IR projectors intended for application within real-time hardware-in-the-loop simulation systems are examined. Attention is paid to the form of the spatial frequency transfer function and, thereby, to the factors that affect the spatial resolution within the composite optical system defined by the IR projector and the imaging unit under test, with particular emphasis on the sampling effects (aliasing) that need to be avoided.
Proc. SPIE. 1969, Infrared Imaging Systems: Design, Analysis, Modeling, and Testing IV
KEYWORDS: Thermography, Signal to noise ratio, Infrared imaging, Photodetectors, Sensors, Black bodies, Infrared radiation, Signal detection, Minimum resolvable temperature difference, Atmospheric optics
The alternative approach presented in a previous conference paper in which spatially dependent expressions were derived for infrared signal and displayed signal-to-noise ratio is extended to reveal the sources of infrared contrast at each pixel site and to the consequent development of methods by which signal and displayed SNR can be calculated across a thermal image.
The three emissive technologies--thin film resistor, silicon bridge resistor and suspended membrane resistor--currently being investigated for dynamic infrared projection applications are thermally compared and contrasted. It is shown that the technologies may be compared quantitatively in terms of both the strong power/speed trade-off that exists and the maximum effective blackbody temperature capability. The suspended membrane technology is shown to be superior in thermal terms, followed by the thin films technology which is found to be practical for high speed small array applications, or for large array applications provided that only small levels of effective blackbody temperature are required. In comparison, it is found that the capability of the silicon bridge technology is constrained in thermal terms both by the comparatively high thermal conductivity of silicon and by the small values of fill factor inherent to the technology.
Thin film resistor technology represents one of the more promising approaches to the problem of providing infrared projection capabilities, particularly when the requirement is for small arrays that operate at high speed at temperatures above 200 degree(s)C. In this paper, the fabrication and characterization of an infrared projector device based on a 2 X 36 thin film resistor array is described. Finite element heat analysis was used during the design phase in order to determine the steady state and transient temperature response of the array. The arrays were fabricated on a polyimide layer on a silicon wafer substrate by using conventional techniques for photolithography, etching, and vacuum deposition. Each resistor element in the array is thermally isolated from its neighboring element by a trench in the polyimide. Characterization of the individual resistors has demonstrated a surface temperature of at least 230 degree(s)C. A transient response of 150 microsecond(s) has been observed for the 10 - 90% rise and fall times.
The output from an infrared projector can be characterized in terms of either output signal radiance or effective blackbody temperature. In this paper, expressions from which both parameters may be derived are developed by spectral analysis of the infrared system comprised of an emissive infrared projector and an imaging unit under test (UUT). Features of the analysis include the separation of the infrared projector characteristics from those of the UUT, the calculation of the output signal radiance in terms of standard blackbody radiation formulae and the inherent algebraic connection that is maintained, enabling the UUT signal current to be readily determined once the output signal radiance is known. A dimensionless graphical procedure is developed for the subsequent determination of the effective blackbody temperature.
The optical aspects of infrared projectors designed specifically for application within real-time hardware-in-the-loop simulation systems are examined. Particular attention is paid to the form of the spatial frequency transfer function and thereby to the factors that affect the spatial resolution within the composite optical system defined by the infrared projector and the imaging unit under test. Expressions are also developed allowing the calculation of both the output signal radiance and effective blackbody temperature for reflective infrared projectors, thus enabling the direct comparison of the radiant output capabilities of projectors of the emissive and reflective types.
The thin film resistor array choice of infrared projector technology is characterized by the comparatively large values of pixel fill factor and emissivity that can be attained but is limited by materials and heat transfer constraints. In this paper, the characteristics and limitations of an infrared projector test device based on a 2 X 25 bilinear thin film nichrome resistor array are described. The steady state and transient performance characteristics have each been assessed by use of both analytical and finite element heat transfer techniques. Test devices based on the design that gave the best predicted performance have been fabricated on a silicon wafer substrate by application of the conventional techniques of photolithography, etching and vacuum deposition, each array being comprised of a patterned polyimide insulation layer sandwiched between the substrate and the nichrome heating elements. Initial characterization experiments have demonstrated a 200 degree(s)C operating temperature capability and 10 - 90% rise and fall times of the order of 100 - 200 microsecond(s) . It is shown that the risetime can be improved significantly by application of a tailored drive voltage waveform, as can the falltime of appropriate thermal connection of the substrate to a low temperature heat sink. Modes of device failure are also discussed.
Infrared analysis is re-examined in the light of the current status of imaging infrared technology. The spectral, spatial and temporal characteristics are all considered from the outset, the analysis being structured such that the sub-system characteristics associated with the infrared scene, transmission, detection and optoelectronic spread processes are separated, wherever possible. Each sub-system may therefore be examined on an essentially independent basis. The radiometric, temporal and spatial contributions are all readily apparent within the separated formulae that are derived for the signal, noise and SNR.
Dynamic infrared scene projection is developing rapidly, driven by the need to simulate imaging infrared systems against complex real-time scenes generated in either a closed-loop or open-loop environment. Applications include simulated operation of imaging infrared missile seekers, search and track systems, thermal imagers and FLIRs, re-entry vehicle trackers, threat warners and infrared fuzes. This course provides a presentation of the status of dynamic infrared scene projection that with a background in infrared imaging technology is designed for users of real-time infrared projection systems.