This work investigates the impact of various types of motion blur on the recognition rate of triangle orientation discrimination (TOD) models. Models based on convolutional neural networks (CNNs) have been proposed as an automated and faster alternative to observer experiments for range performance assessment. They may also give insights into the impact of system degradations on the performance of automated target recognition algorithms. However, the effects of many image distortions on the recognition rate of such models are relatively unknown. The recognition rate of CNN-based TOD models is examined in terms of different forms of motion blur, such as jitter, linear and sinusoidal motion. For model training and validation, simulated images are used. Triangles with four directions and different sizes, positions are used as targets, which are superposed on natural images as background taken from the image database “Open Images V7”. Motion blur of varying strength is applied to both the triangle and the entire image to simulate movements of the target and imager. Additionally, common degradation effects of imagers are applied, such as white sensor noise and blur due to diffraction and detector footprint. The recognition rates of the models are compared for target motion and global motion as well as for the different motion types. Furthermore, dependencies of the recognition rate on blur strength, triangle size and noise level are shown. The study shows interrelationships and differences between target motion and global motion regarding TOD classifications. The inclusion of motion blur in training can also increase model accuracy in validation. These findings are crucial for range performance assessment of thermal imagers for fast-moving targets.
KEYWORDS: Signal to noise ratio, Education and training, Data modeling, Detection and tracking algorithms, Image enhancement, Databases, RGB color model, Optical engineering, Forward looking infrared, Performance modeling
Within the last decades, a large number of techniques for contrast enhancement has been proposed. There are some comparisons of such algorithms for few images and figures of merit. However, many of these figures of merit cannot assess usability of altered image content for specific tasks, such as object recognition. In this work, the effect of contrast enhancement algorithms is evaluated by means of the triangle orientation discrimination (TOD), which is a current method for imager performance assessment. The conventional TOD approach requires observers to recognize equilateral triangles pointing in four different directions, whereas here convolutional neural network models are used for the classification task. These models are trained by artificial images with single triangles. Many methods for contrast enhancement highly depend on the content of the entire image. Therefore, the images are superimposed over natural backgrounds with varying standard deviations to provide different signal-to-background ratios. Then, these images are degraded by Gaussian blur and noise representing degradational camera effects and sensor noise. Different algorithms, such as the contrast-limited adaptive histogram equalization or local range modification, are applied. Then accuracies of the trained models on these images are compared for different contrast enhancement algorithms. Accuracy gains for low signal-to-background ratios and sufficiently large triangles are found, whereas impairments are found for high signal-to-background ratios and small triangles. A high generalization ability of our TOD model is found from the similar accuracies for several image databases used for backgrounds. Finally, implications of replacing triangles with real target signatures when using such advanced digital signal processing algorithms are discussed. The results are a step toward the assessment of those algorithms for generic target recognition.
KEYWORDS: Signal to noise ratio, Data modeling, Detection and tracking algorithms, Sensors, Image processing, Cameras, RGB color model, Visual process modeling, Imaging systems, Image contrast enhancement
A current approach for performance assessment of imagers is triangle orientation discrimination (TOD). This approach requires observers or human visual system (HVS) models to recognize equilateral triangles pointing in four different directions. Imagers may apply embedded advanced digital signal processing (ADSP) for contrast enhancement, noise reduction, edge sharpening, etc. Unfortunately, applied methods are in general not documented and hence unknown. Within the last decades a vast amount of techniques for contrast enhancement has been proposed. There are some comparisons of such algorithms for few images and figures of merit. However, many of these figures of merit cannot assess usability of altered image content for specific tasks such as object recognition In this work different algorithms for contrast enhancement are compared in terms of TOD assessments by convolutional neural networks (CNN) as models. These models are trained by artificial images with single triangles. Many methods for contrast enhancement highly depend on the content of the entire image. Therefore, the images are superimposed by natural backgrounds with varying standard deviations to provide different signal-to-background ratios. Then these images are degraded by Gaussian blur and noise representing degradational camera effects and sensor noise. Different algorithms are applied, such as the contrast-limited adaptive histogram equalization or local range modification. Then accuracies of the trained models on these images are compared for different ADSP algorithms. Accuracy gains for low signal-to-background ratio and sufficiently large triangles are found, while impairments are found for high signal-to-background ratio and small triangles. Finally, implications of replacing triangles by real target signatures when using such ADSP algorithms are discussed. The results can be a step towards the assessment of those algorithms for generic target recognition.
This paper presents experimental investigations on active compressive sensing imaging through turbulence. We developed a laboratory testbed in which different compressive sensing configurations have been tested under various turbulence conditions. Series of images of a target were acquired and analyzed using three different metrics. The measurements have been performed under continuous-wave laser illumination at 635 nm.
We present a model that calculates the reflected intensity of a high-energy laser irradiating a metallic target. It will enable us to build a laser safety model that can be used to determine nominal ocular hazard distances for high-energy laser engagements. The reflection was first measured in an experiment at 2 m distance from the target. After some irradiation time, the target begins to melt and the reflected intensity presents intensity patterns composed of caustics, which vary rapidly and are difficult to predict. A specific model is developed that produces similar caustic patterns at 2 m distance and can be used to calculate the reflected intensity at arbitrary distances. This model uses a power spectral density (PSD) to describe the melting metal surface. From this PSD, a phase screen is generated and applied onto the electric field of the laser beam, which is then propagated to a distance of 2 m. The simulated intensity distributions are compared to the measured intensity distributions. To quantify the similarity between simulation and experiment, different metrics are investigated. These metrics were chosen by evaluating their correlation with the input parameters of the model. An artificial neural network is then trained, validated and tested on simulated data using the aforementioned metrics to find the input parameters of the PSD that lead to the most similar caustics. Additionally, we tested another approach based on an autoencoder, which was tested on the MNIST dataset, to eventually generate a phase screen directly by using the caustics images.
Atmospheric turbulence often limits the performance of long-range imaging systems in applications. Realistic turbulence simulations provide means to evaluate this effect and assess turbulence mitigation algorithms. Current methods typically use phase screens or turbulent point spread functions (PSFs) to simulate the image distortion and blur due to turbulence. While the first method takes long computation times, the latter requires empirical models or libraries of PSF shapes and their associated tip and tilt motion, which might be overly simplistic for some applications. In this work, an approach is evaluated which tries to avoid these issues. Generative neural networks models are able to generate extremely realistic imitations of real (image) data with a short calculation time. To treat anisoplanatic imaging for the considered application, the model output is an imitation PSF-grid that has to be applied to the input image to yield the turbulent image. Certain shape features of the model outcome can be controlled by traversing within subsets of the model input space or latent space. The use of a conditional variational autoencoder (cVAE) appears very promising to yield fast computation times and realistic PSFs and is therefore examined in this work. The cVAE is trained on field trial camera images of a remote LED array. These images are considered as grids of real PSFs. First the images are pre-processed and their PSFs properties are determined for each frame. Main goal of the cVAE is the generation of PSF-grids under conditional properties, e.g. moments of PSFs. Different approaches are discussed and employed for a qualitative evaluation of the realism of the PSF-grids generated by the trained models. A comparison of required simulation computing time is presented and further considerations regarding the simulation method are discussed.
Air turbulence can be a major impairment source for long-range imaging applications. There is great interest in the assessment of turbulence mitigation techniques based on machine learning models. In general such models require lots of image data for robust training and validation. Experimental acquisition of image data in field trials is time-consuming and environmental conditions such as daytime and weather cannot be specifically controlled. Several methods for turbulence simulation have been proposed in recent years. Many of these are based on phase screens or models turbulent point spread functions (PSFs). Often simple turbulence models such as the Kolmogorov or Von Karman spectrum are used. Therefore these methods cannot provide insight in the influence and relevance of other turbulence parameters such as inner scale and (non-)Kolmogorov power slope. In this work a data fitting procedure for the determination of turbulence model parameters based on experimental data is shown. Hereby the Generalized modified Von Karman spectrum (GMVKS) is used. Differential tilt variances (DTV) are calculated from centroid displacements in video sequences of a recorded LED grid. Then the experimental data is fitted to theoretical expressions of DTV by numerical integration over the turbulence model. Image data was acquired in field trials on several days at the same location. Then a beam propagation method using Markov GMVKS phase screens with determined model parameters is used to generate a grid of PSF images which represent degradation for different viewing angles. For validation, DTVs based on centroid displacements of the simulated PSFs are calculated and compared with the corresponding measured data of LED centroid displacements and theoretical data. Cumulative distribution functions of the model parameters for all recording dates are provided to show the diversity of turbulence conditions. These can be used as prior knowledge for future turbulence simulations to include various model parameters and hence different conditions of image degradation. Finally the extensibility of the data fitting approach to other turbulence spectra, e.g. anisotropic spectra, is discussed.
Remote detection of vibrational features from an object is important for many short-range civil applications, but it is also of interest for long-range applications in the defense and security areas. The well-established laser Doppler vibrometry technique is widely used as a high-sensitivity, noncontact method. The development of camera technology in recent years made image-based methods reliable passive alternatives for vibration and dynamic measurements. We investigate and discuss the potential of high-speed imaging technique for medium- and long-range vibration detection. The sensitivity and the limitations of the method are experimentally investigated in comparison to the well-established Doppler vibrometry technique. As atmospheric turbulence is expected to become a limiting factor for long-range applications, imaging in the short-wave IR (SWIR) to mid-wave IR (MWIR) rather than in the visual range is advantageous due to the longer wavelength. We present experiments on the vibration detection from SWIR and MWIR image sequences, as well as additional experiments on the extraction of vibration signature under strong local turbulence conditions.
A large variety of image quality metrics has been proposed within the last decades. The majority of these metrics has been investigated only for single image degradations like noise, blur and compression on limited sets of domain-specific images. For assessing imager performance, however, a task-specific evaluation of captured imagers with user-defined content seems, in general, more appropriate than using such metrics. This paper presents an approach to image quality assessment of camera data by comparison of classification rates of models individually trained to solve simple classification tasks on images containing single geometric primitives. Examples of considered tasks are triangle orientation discrimination or the determination of number of line pairs for bar targets. In order to make models more robust against image degradations typically occurring in real cameras, data augmentation is applied on pristine imagery of geometric primitives in the training phase. Pristine imagery is impaired by a variety of simulated image degradations, e.g. Gaussian noise, salt and pepper noise for defective pixels, Gaussian and motion blur, perspective image distortion. The trained models are then applied to real camera images and classification rates are calculated for geo- metric primitives of different sizes, contrasts and center positions. For task-related performance ranking, these classification rates could be compared for multiple cameras or camera settings. An advantage of this approach is that the amount of training data is practically inexhaustible due to artificial imagery and applied image degradations, which makes it easy to counteract model over fitting by increasing the number of considered realizations of image degradations applied to the imagery and hence increasing the variability of training data.
Atmospheric turbulence degrades focal-plane-array (FPA) camera images because of intensity fluctuation, distortion, and blur, notably for long-range applications. Compressive sensing (CS) imaging techniques use series of measurements whose temporal and spatial characteristics differ from those of conventional FPA systems. The paper discusses how turbulence affects the SWIR image quality using both CS techniques and a conventional InGaAs FPA camera.
ECOMOS is a multinational effort within the framework of an EDA Project Arrangement. Its aim is to provide a generally accepted and harmonized European computer model for computing nominal Target Acquisition (TA) ranges of optronic imagers operating in the Visible or thermal Infrared (IR). The project involves close co-operation of defense and security industry and public research institutes from five nations: France, Germany, Italy, The Netherlands and Sweden. ECOMOS will use and combine existing European tools, to build up a strong competitive position. In Europe, there are two well-accepted approaches for providing TA performance data: the German TRM (Thermal Range Model) model and the Netherlands TOD (Triangle Orientation Discrimination) method. ECOMOS will include both approaches. The TRM model predicts TA performance analytically, whereas the TOD prediction model utilizes the TOD test method, imaging simulation and a Human Visual System model in order to assess device performance. For the characterization of atmosphere and environment, ECOMOS uses the French model and software MATISSE (Modélisation Avancée de la Terre pour l'Imagerie et la Simulation des Scènes et de leur Environnement). The first software implementation of ECOMOS has been finalized in spring 2019. In this presentation, the key features implemented in the current version are elucidated. In addition, the final ECOMOS software structure as well as an overview of the user guidance within ECOMOS are shown.
Compressive sensing (CS) is an imaging method that enables the replacement of expensive matrix detectors by small and cheap detectors with one or a few detector elements. A high-resolution image is realized from a series of individual single-value measurements. Each measurement consists of capturing the image from an object or a scene after coding by a well-defined pattern. The reconstruction of the high-resolution image requires a number of measurements significantly smaller than the number of full-frame image pixels. This is because most natural images may be sparsely coded, i.e. we may find an appropriate basis for which most coefficients are close to zero. This paper reports CS experiments under pulse laser illumination at 1.55 μm. The light collected from the observed scene is spatially modulated using a digital micromirror device (DMD) and projected onto a single-pixel detector. The applied binary patterns are generated using a Hadamard matrix. Different approaches for pattern selection have been implemented and compared.
Based on previous work on thermal imager performance analysis at Fraunhofer IOSB using specific scenes and patterns, we present our advances in setting up a testbed for thermal imager characterization with a MIRAGE™ XL infrared scene projector.
In the first part, we outline the experimental setup of our testbed. It allows for mimicking infrared imaging of real scenes in a controlled laboratory environment. We describe the process of dynamic infrared scene generation as well as the physical limitations of our scene projection setup.
A second part discusses ongoing and future applications. This testbed extends our standard lab measurements for thermal imagers by a image based performance analysis method. Scene based methods are necessary to investigate and assess advanced digital signal processing (ADSP) algorithms which are becoming an integral part of thermal imagers. We use this testbed to look into inferences of unknown proprietary ADSP algorithms by choosing suitable test scenes.
Furthermore, we investigate the influence of dazzling on thermal imagers by coupling infrared laser radiation into the projected scene. The studies allow to evaluate the potential and hazards of infrared dazzling and to describe correlated effects. In a future step, we want to transfer our knowledge of VIS/NIR laser protection into the infrared regime.
ECOMOS is a multinational effort within the framework of an EDA Project Arrangement. Its aim is to provide a generally accepted and harmonized European computer model for computing nominal Target Acquisition (TA) ranges of optronic imagers operating in the Visible or thermal Infrared (IR). The project involves close co-operation of defense and security industry and public research institutes from France, Germany, Italy, The Netherlands and Sweden. ECOMOS uses two approaches to calculate Target Acquisition (TA) ranges, the analytical TRM4 model and the image-based Triangle Orientation Discrimination model (TOD).
In this paper the IR imager simulation tool, Optronic System Imaging Simulator (OSIS), is presented. It produces virtual camera imagery required by the TOD approach. Pristine imagery is degraded by various effects caused by atmospheric attenuation, optics, detector footprint, sampling, fixed pattern noise, temporal noise and digital signal processing. Resulting images might be presented to observers or could be further processed for automatic image quality calculations.
For convenience OSIS incorporates camera descriptions and intermediate results provided by TRM4. For input OSIS uses pristine imagery tied with meta information about scene content, its physical dimensions, and gray level interpretation. These images represent planar targets placed at specified distances to the imager.
Furthermore, OSIS is extended by a plugin functionality that enables integration of advanced digital signal processing techniques in ECOMOS such as compression, local contrast enhancement, digital turbulence mitiga- tion, to name but a few. By means of this image-based approach image degradations and image enhancements can be investigated, which goes beyond the scope of the analytical TRM4 model.
In this paper we introduce a software tool for image based computer simulation of an underwater gated viewing system. This development is helpful as a tool for the discussion of a possible engagement of a gated viewing camera for underwater imagery. We show the modular structure of implemented input parameter sets for camera, laser and environment description and application examples of the software tool. The whole simulation includes the scene illumination through a laser pulse with its energy pulse form and length as well as the propagation of the light through the open water taking into account complex optical properties of the environment. The scene is modeled as a geometric shape with diverse reflective areas and optical surface properties submerged in the open water. The software is based on a camera model including image degradation due to diffraction, lens transmission, detector efficiency and image enhancement by digital signal processing. We will show simulation results on some example configurations. Finally we will discuss the limits of our method and give an outlook to future development.
A few image quality metrics for blur assessment have been presented in the last years. However, most of those metrics do not take image noise into account. Yet, image noise is an unavoidable part of the image forming process with digital cameras. Some thermal imagers show larger sensor noise and inhomogeneity compared to cameras operating in the visible range. Further, natural imagery might contain a combination of several degradations. Assessment of degraded images by observer trials is expensive and time consuming. A single robust quality metric might be derived by metrics highly responsive to single degradations and insensitive to others. Hence separate assessment of image blur and noise seems to be reasonable. In this paper we present a deep learning approach for noise-insensitive blur predictions by using Convolutional Neural Networks (CNN) on image patches. In contrast to current blur metrics the model output is highly correlated to blur distortion over a wide range of image noise. The model is trained on images of ImageNet database impaired by Gaussian blur and noise and tested on artificial and natural image data. Local blur estimation based on patches is especially useful for estimation of non-uniform blur due to motion and atmospheric turbulence.
Due to advances in technology, modern thermal imagers resemble sophisticated image processing systems in functionality. Advanced signal and image processing tools enclosed into the camera body extend the basic image capturing capability of thermal cameras. This happens in order to enhance the display presentation of the captured scene or specific scene details. Usually, the implemented methods are proprietary company expertise, distributed without extensive documentation. This makes the comparison of thermal imagers especially from different companies a difficult task (or at least a very time consuming/expensive task - e.g. requiring the execution of a field trial and/or an observer trial). For example, a thermal camera equipped with turbulence mitigation capability stands for such a closed system. The Fraunhofer IOSB has started to build up a system for testing thermal imagers by image based methods in the lab environment. This will extend our capability of measuring the classical IR-system parameters (e.g. MTF, MTDP, etc.) in the lab. The system is set up around the IR- scene projector, which is necessary for the thermal display (projection) of an image sequence for the IR-camera under test. The same set of thermal test sequences might be presented to every unit under test. For turbulence mitigation tests, this could be e.g. the same turbulence sequence. During system tests, gradual variation of input parameters (e. g. thermal contrast) can be applied. First ideas of test scenes selection and how to assembly an imaging suite (a set of image sequences) for the analysis of imaging thermal systems containing such black boxes in the image forming path is discussed.
KEYWORDS: High speed imaging, Vibrometry, Doppler effect, Cameras, High speed cameras, Video, Sensors, Signal detection, Detection and tracking algorithms, Information visualization
The development of camera technology in recent years has made high speed imaging a reliable method in vibration and dynamic measurements. The passive recovery of vibration information from high speed video recordings was reported in several recent papers. A highly developed technique, involving decomposition of the input video into spatial subframes to compute local motion signals, allowed an accurate sound reconstruction. A simpler technique based on image matching for vibration measurement was also reported as efficient in extracting audio information from a silent high speed video. In this paper we investigate and discuss the sensitivity and the limitations of the high speed imaging technique for vibration detection in comparison to the well-established Doppler vibrometry technique. Experiments on the extension of the high speed imaging method to longer range applications are presented.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.