Suspicious human behaviors can be defined by the user, and in long distance imaging it may include bending the body during walking or crawling, in contrast to regular walking for instance. State-of-the-art methods using convolutional neural networks (CNNs) dealt in general with “clean” signals, in which the object of interest is relatively close to the camera, and therefore fairly clear and easily distinguished from the surrounding environment. This makes it easier to capture detailed information regarding the object and its action. However, in relatively long distance imaging (few kilometers and above) additional difficulties occur which affect the performances of these tasks, since the captured videos are likely to be degraded by the atmospheric path that cause blur and spatiotemporal-varying distortions. Both of these degradation types may reduce the ability for action recognition. These effects become more significant for longer imaging distances and smaller sizes of the objects of interest in the image. The images of objects in imaging through long distance are usually relatively small, and hence, the range of actions that can be resolved is more limited, particularly under strong atmospheric effects. In this study, we perform action localization by first applying optical flow unique processing, and also using a variant of SSD (Single Shot MultiBox Detector) to regress and classify detection boxes in each video frame potentially containing an action of interest.
Surveillance in long-distance turbulence-degraded video is a difficult challenge because of the effects of the atmospheric turbulence that causes blur and random shifts in the image. As imaging distances increase, the degradation effects become more significant. This paper presents a method for surveillance in long-distance turbulence-degraded videos. This method is based on employing new criteria for discriminating true from false object detections. We employ an adaptive thresholding procedure for background subtraction, and implement new criteria for distinguishing true from false moving objects, that take into account the temporal consistency of both shape and motion properties. Results show successful detection also tracking of moving objects on challenging video sequences, which are significantly distorted with atmospheric turbulence. However, when the imaging distance is increased higher false alarms may occur. The method presented here is relatively efficient and has low complexity.
Classification of moving objects in imaging through long-distance atmospheric path may be affected by distortions such as blur and spatiotemporal movements caused by air turbulence. This work aims to study and quantify the effects of these distortions on the ability to classify moving objects in atmospherically degraded video signals. For this purpose, we perform simulations and examine real long-range thermal video cases. In the simulation, we evaluate various geometrical (shape-based) object features for classification at different distortion levels. Furthermore, we examine the influence of image restoration on the classification performances in the real-degraded videos, using geometrical and textural features (combined and in separate) of the objects. Principal component analysis together with both k-nearest neighbor and support vector machines is used for the classification process. Results show how classification performances decrease as the level of blur increases, and how successful digital image restoration for real cases can significantly improve the classification performances.
Acquisition and classification of moving objects in imaging through long-distance atmospheric path (more than 1-2 km)
may be affected by distortions such as blur and spatiotemporal movements caused by air turbulence. These distortions
are more meaningful when the size of the objects is relatively small (for instance, few pixels width). This work aims to
study and quantify the effects of these distortions on the ability to classify small moving objects in atmosphericallydegraded
video signals. For this purpose, moving objects were extracted from real video signals recorded through longdistance
atmospheric path. Then, various geometrical and textural object features were extracted, and reduced to two
principle components using principle component analysis (PCA). The effect of the atmospheric distortion on object
classification was examined using support vector machine (SVM) classifier. Furthermore, the influence of image
restoration on the classification performances was examined for the real-degraded videos. Results show how
classification performances are decreasing when the images are degraded by the atmospheric path compared to the case
where successful image restoration is performed.
Proc. SPIE. 6737, Electro-Optical and Infrared Systems: Technology and Applications IV
KEYWORDS: Thermography, Point spread functions, Detection and tracking algorithms, Image processing, Video, Image restoration, Video surveillance, Motion detection, Modulation transfer functions, Binary data
Automatic acquisition of moving objects from long-distance video sequence is a fundamental task in many applications
such as surveillance and reconnaissance. However, the atmospheric degradations, which include blur and
spatiotemporal-varying distortions, may reduce the quality of such videos, and therefore, the ability to acquire moving
targets automatically. Pervious studies in the field of automatic acquisition of moving objects ignored the blur in the
video frames. They usually employed simple methods for noise reduction (such as temporal and spatial smoothing) and
motion compensation (registration of frames). The purpose of this work is to determine the effect of image restoration
(de-blurring) on the ability to acquire moving objects (such as humans and vehicles) automatically. This is done here by
first, restoring the long-distance thermal videos using a novel blind image deconvolution method developed recently, and
then comparing the automatic acquisition capabilities in the restored videos versus the non-restored versions. Results
show that image restoration can significantly improve the automatic acquisition capability. These results correspond to a
previous study which demonstrated that image restoration can significantly improve the ability of human observers to
acquire moving objects from a long-range thermal video.
Remotely sensed videos, captured by high-resolution imagers, are likely to be degraded by the atmosphere. In still images, the degradation sources, which include turbulence and aerosols, mainly cause blur. In video sequences, however, spatiotemporally varying distortions caused by turbulence also become important. These atmospheric degradations reduce image quality and therefore the ability of target acquisition by the observers. The effects of image quality and image restoration (deblurring) on target acquisition in still images were examined previously in several studies. Nevertheless, results obtained in static situations may not be appropriate for dynamic situations (with moving targets), which are frequently more realistic. This work examines the effect of image restoration on the ability of observers to acquire moving objects (such as humans and vehicles) in video sequences. This is done through perception experiments that compare acquisition probabilities in both restored and nonrestored video sequences captured by a remote-sensing thermal imaging system. Results show that image restoration can significantly improve the acquisition probability. These results correspond to the static case. However, unlike the static case, considerably smaller differences were obtained here between the probabilities of target detection and target recognition.
Various applications such as industrial product inspection or low signal-to-noise situations (as in thermal imaging) employ a time delay and integration (TDI) scanning imaging technique. Due to common vibration sources such as the camera platform motion or the thermal detector's cooling system, the acquired image may be degraded by severe shift-variant geometric distortions and motion blur. We use these vibrations in terms of superresolution to create an improved high-resolution video sequence from the degraded lower resolution sequence, in two main stages: subpixel motion estimation with respect to translations and rotations, used for point spread function (PSF) estimation, followed by an efficient implementation of the projection onto convex sets (POCS) method. We generalize and considerably improve a previous technique for restoration of a single image captured by a translational vibrated staggered-TDI camera (Hochman et al., 2004). The proposed method is implemented with both simulated videos and real degraded thermal videos. A comparative analysis shows an advantage of the proposed method over others in restoring the vibrated videos.
Time-delay and integration (TDI) scanning imaging technique is used in various applications such as military reconnaissance and industrial product inspection. Its high sensitivity is significant in low light-level imaging and in thermal imaging. Due to physical constraints the TDI sensor elements may have a staggered structure, in which the odd and the even sensors are horizontally separated. The electrical cooling system of the detectors, as well as the camera platform, vibrates the system and causes image distortions such as space variant comb effects and motion blur. These vibrations are utilized here in means of superresolution in order to create an improved high resolution sequence from lower resolution sequence in two main stages: inter-frame space-variant motion estimation followed by an efficient implementation of the projection onto convex sets (POCS) restoration method. This work generalizes an algorithm for restoration of a single staggered TDI image preformed previously. The additional information included in an image sequence allows more efficient restoration process, and better restoration results. The lack of any assumption about the correlation between the vibrations of the odd and even sensors enables also an implementation to TDI cameras that don't contain the staggering structure. Experimental results with real degraded thermal video are provided.