Hyperspectral imaging has a wide range of applications relying on remote material identification, including astronomy, mineralogy, and agriculture; however, due to the large volume of data involved, the complexity and cost of hyperspectral imagers can be prohibitive. The exploitation of redundancies along the spatial and spectral dimensions of a hyperspectral image of a scene has created new paradigms that overcome the limitations of traditional imaging systems. While compressive sensing (CS) approaches have been proposed and simulated with success on already acquired hyperspectral imagery, most of the existing work relies on the capability to simultaneously measure the spatial and spectral dimensions of the hyperspectral cube. Most real-life devices, however, are limited to sampling one or two dimensions at a time, which renders a significant portion of the existing work unfeasible. We propose a new variant of the recently proposed serial hybrid vectorial and tensorial compressive sensing (HCS-S) algorithm that, like its predecessor, is compatible with real-life devices both in terms of the acquisition and reconstruction requirements. The newly introduced approach is parallelizable, and we abbreviate it as HCS-P. Together, HCS-S and HCS-P comprise a generalized framework for hybrid tenso-vectorial compressive sensing, or HCS for short. We perform a detailed analysis that demonstrates the uniqueness of the signal reconstructed by both the original HCS-S and the proposed HCS-P algorithms. Last, we analyze the behavior of the HCS reconstruction algorithms in the presence of measurement noise, both theoretically and experimentally.

Reassembling fragmented image files plays a crucial role in seizing digital evidence from scattered digital image files. The existing algorithms are mainly graph based, which cast the reassembly problem as a K-vertex disjoint path problem in a directed complete graph, which is an NP-complete problem. Based on the padding bytes in BMP files, we present a method to exclude most impossible paths, which can improve the accuracy and decrease the time complexity of the existing graph-based methods. According to the alignment rule of BMP format, padding bytes must be appended to the end of each row to bring up the length of the row to a multiple of 4 bytes. Hence the fragment, being a vertex of the path which correctly reassembles a file, has a property; its byte values at padding positions must be the padding values. Only the fragments with such property can be candidate fragments for the vertex. On the test dataset which is constructed based on 330 image files, taking eight classical methods as examples, we show that the proposed method produces an accuracy improvement ranging from 32% to 55%, and reduces the run time to a scope from 1/6 to 1/237.

We investigate the extension of the recently proposed weighted Fourier burst accumulation (FBA) method into the wavelet domain. The purpose of FBA is to reconstruct a clean and sharp image from a sequence of blurred frames. This concept lies in the construction of weights to amplify dominant frequencies in the Fourier spectrum of each frame. The reconstructed image is then obtained by taking the inverse Fourier transform of the average of all processed spectra. We first suggest replacing the rigid registration step used in the original algorithm with a nonrigid registration in order to process sequences acquired through atmospheric turbulence. Second, we propose to work in a wavelet domain instead of the Fourier one. This leads us to the construction of two types of algorithms. Finally, we propose an alternative approach to replace the weighting idea by an approach promoting the sparsity in the used space. Several experiments are provided to illustrate the efficiency of the proposed methods.

We propose a frame rate upconversion (FRUC) method using pyramid structures (PS) and dense motion vector fields (MVFs). In FRUC processes, performance is dominantly dependent on motion compensation, thus motion vectors (MVs) must be precisely estimated. Variable sizes of blocks and large search ranges are needed to estimate the MVs of large objects and large movements; however, we use PS and dense MVFs to estimate MVs for various conditions. In the PS, we first estimate MVs on level 0, which is the most reduced image in the PS (L-1 times downsampling), and MVs on the high levels are estimated except for pixels having large corresponding MVs on the lower levels. Integration of MVFs for all levels is followed by a vector median filter to remove noises. Finally, a motion compensated frame is interpolated by weight-overlapped block motion compensation.

A two-step face-hallucination framework is proposed to reconstruct a high-resolution (HR) version of a face from an input low-resolution (LR) face, based on learning from LR–HR example face pairs using orthogonal canonical correlation analysis (orthogonal CCA) and linear mapping. In the proposed algorithm, face images are first represented using principal component analysis (PCA). Canonical correlation analysis (CCA) with the orthogonality property is then employed, to maximize the correlation between the PCA coefficients of the LR and the HR face pairs to improve the hallucination performance. The original CCA does not own the orthogonality property, which is crucial for information reconstruction. We propose using orthogonal CCA, which is proven by experiments to achieve a better performance in terms of global face reconstruction. In addition, in the residual-compensation process, a linear-mapping method is proposed to include both the inter- and intrainformation about manifolds of different resolutions. Compared with other state-of-the-art approaches, the proposed framework can achieve a comparable, or even better, performance in terms of global face reconstruction and the visual quality of face hallucination. Experiments on images with various parameter settings and blurring distortions show that the proposed approach is robust and has great potential for real-world applications.

In many practical applications, we frequently face the awkward problem in which an image classifier trained in a scenario is difficult to use in a new scenario. Traditionally, the probability inference-based methods are used to solve this problem. From the point of image representation, we propose an approach for domain adaption of image classification. First, all source samples are supposed to form the dictionary. Then, we encode the target sample by combining this dictionary and the local geometric information. Based on this new representation, called target nearest-neighbor representation, image classification can obtain good performance in the target domain. Our core contribution is that the nearest-neighbor information of the target sample is technically exploited to form more robust representation. Experimental results confirm the effectiveness of our method.

Cameras mounted on scouting vehicles frequently suffer from image shake because of unintentional motions. Image shake is a main source of inaccuracies that lead to bad scouting results, particularly in mountain areas with complicated terrains. To overcome this disadvantage, this study proposes a digital image stabilization method based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and structural similarity (SSIM) to generate a stable scouting video sequence. The proposed method first calculates the global motion vector (GMV) from a scouting video sequence using the bit-plane matching algorithm. To separate jitter motion from intentional motion, we decompose GMV into several modes using CEEMDAN. Then according to different structural characteristics, SSIM is used to draw a boundary among modes to separate jitter motion from intentional motion. To evaluate stabilization performance in complicated situations, several known methods and the proposed stabilization method are compared. Experimental results show that CEEMDAN outperforms the other stabilization methods under mountain area conditions.

Appearance-based bag-of-visual words (BoVW) models are employed to represent the frequency of a vocabulary of local features in an image. Due to their versatility, they are widely popular, although they ignore the underlying spatial context and relationships among the features. Here, we present a unified representation that enhances BoVWs with explicit local and global structure models. Three aspects of our method should be noted in comparison to the previous approaches. First, we use a local structure feature that encodes the spatial attributes between a pair of points in a discriminative fashion using class-label information. We introduce a bag-of-structural words (BoSW) model for the given image set and describe each image with this model on its coarsely sampled relevant keypoints. We then combine the codebook histograms of BoVW and BoSW to train a classifier. Rigorous experimental evaluations on four benchmark data sets demonstrate that the unified representation outperforms the conventional models and compares favorably to more sophisticated scene classification techniques.

Palmprint recognition systems that use three-dimensional (3-D) information of the palm surface are the most recently explored techniques to overcome some two-dimensional palmprint difficulties. These techniques are based on light structural imaging. In this work, a 3-D ultrasound palmprint recognition system is proposed and evaluated. Volumetric images of a region of the human hand are obtained by moving an ultrasound linear array along its elevation direction and one by one acquiring a number of B-mode images, which are then grouped in a 3-D matrix. The acquisition time was contained in about 5 s. Much information that can be exploited for 3-D palmprint recognition is extracted from the ultrasound volumetric images, including palm curvature and other under-skin information as the depth of the various traits. The recognition procedure developed in this work is based on the analysis of the principal curvatures of palm surface, i.e., mean curvature image, Gaussian curvature image, and surface type. The proposed method is evaluated by performing verification and identification experiments. Preliminary results have shown that the proposed system exhibits an acceptable recognition rate. Further possible improvements of the proposed technique are finally highlighted and discussed.

Passive forgery detection aims to detect traces of image tampering without the need for prior information. With the increasing demand for image content protection, passive detection methods able to identify image tampering areas are increasingly needed. However, most current passive approaches either work only for image-level JPEG compression detection and cannot localize region-level forgery, or suffer from high-false detection rates in localizing altered regions. This paper proposes an effective approach based on discrete cosine transform coefficient analysis for the detection and localization of altered regions of JPEG compressed images. This approach can also work with altered JPEG images resaved in JPEG compressed format with different quality factors. Experiments with various tampering methods such as copy-and-paste, image completion, and composite tampering, show that the proposed approach is able to effectively detect and localize altered areas and is not sensitive to image contents such as edges and textures.

Variational methods have attracted a lot of attention in the literature, especially for image and mesh segmentation. The methods aim at minimizing the energy to optimize both edge and region detections. We propose a spectral mesh decomposition algorithm to obtain disjoint but meaningful regions of an input mesh. The related optimization problem is nonconvex, and it is very difficult to find a good approximation or global optimum, which represents a challenge in computer vision. We propose an alternating split Bregman algorithm for mesh segmentation, where we extended the image-dedicated model to a three-dimensional (3-D) mesh one. By applying our scheme to 3-D mesh segmentation, we obtain fast solvers that can outperform various conventional ones, such as graph-cut and primal dual methods. A consistent evaluation of the proposed method on various public domain 3-D databases for different metrics is elaborated, and a comparison with the state-of-the-art is performed.

Images taken under underwater condition usually have color cast and serious loss of contrast and visibility. Degraded underwater images are inconvenient for observation and analysis. In order to address these problems, an underwater image-enhancement method is proposed. A simple yet effective underwater image color cast removal algorithm is first presented based on the optimization theory. Then, based on the minimum information loss principle and inherent relationship of medium transmission maps of three color channels in an underwater image, an effective visibility restoration algorithm is proposed to recover visibility, contrast, and natural appearance of degraded underwater images. To evaluate the performance of the proposed method, qualitative comparison, quantitative comparison, and color accuracy test are conducted. Experimental results demonstrate that the proposed method can effectively remove color cast, improve contrast and visibility, and recover natural appearance of degraded underwater images. Additionally, the proposed method is comparable to and even better than several state-of-the-art methods.

Multimodal imaging systems have recently been drawing attention in fields such as medical imaging, remote sensing, and video surveillance systems. In such systems, estimating depth has become possible due to the promising progress of multimodal matching techniques. We perform a systematic performance evaluation of similarity measures frequently used in the literature for dense multimodal stereovision. The evaluated measures include mutual information (MI), sum of squared distances, normalized cross-correlation, census transform, local self-similarity (LSS) as well as descriptors adopted to multimodal settings, like scale invariant feature transform (SIFT), speeded-up robust features (SURF), histogram of oriented gradients (HOG), binary robust independent elementary features, and fast retina keypoint (FREAK). We evaluate the measures over datasets we generated, compiled, and provided as a benchmark and compare the performances using the Winner Takes All method. The datasets are (1) synthetically modified four popular pairs from the Middlebury Stereo Dataset (namely, Tsukuba, Venus, Cones, and Teddy) and (2) our own multimodal image pairs acquired using the infrared and the electro-optical cameras of a Kinect device. The results show that MI and HOG provide promising results for multimodal imagery, and FREAK, SURF, SIFT, and LSS can be considered as alternatives depending on the multimodality level and the computational complexity requirements of the intended application.

A skew-estimation method using straight lines in document images is presented. Unlike conventional approaches exploiting the properties of text, we formulate the skew-estimation problem as an estimation task using straight lines in images and focus on robust and accurate line detection. To be precise, we adopt a block-based edge detector followed by a progressive line detector to take clues from a variety of sources such as text lines, boundaries of figures/tables, vertical/horizontal separators, and boundaries of textblocks. Extensive experiments on the datasets of skewed images and competition results reveal that the proposed method works robustly and yields accurate skew-estimation results.

The bag-of-visual-words (BOVW) and Fisher kernel are two popular models in human action recognition, and support vector machine (SVM) is the most commonly used classifier for the two models. We show two kinds of group structures in the feature representation constructed by BOVW and Fisher kernel, respectively, since the structural information of feature representation can be seen as a prior for the classifier and can improve the performance of the classifier, which has been verified in several areas. However, the standard SVM employs L2-norm regularization in its learning procedure, which penalizes each variable individually and cannot express the structural information of feature representation. We replace the L2-norm regularization with group lasso regularization in standard SVM, and a group lasso regularized-support vector machine (GLRSVM) is proposed. Then, we embed the group structural information of feature representation into GLRSVM. Finally, we introduce an algorithm to solve the optimization problem of GLRSVM by alternating directions method of multipliers. The experiments evaluated on KTH, YouTube, and Hollywood2 datasets show that our method achieves promising results and improves the state-of-the-art methods on KTH and YouTube datasets.

Limited resolution, blurring, warping, and additive noise associated with image acquisition and storage often make the barcode image degraded, generating low-resolution (LR) barcode images. Barcodes in degraded LR images are difficult to recognize. The goal of this paper is to introduce the potential of super-resolution (SR) technique in conquering the aforementioned challenges, and a variational Bayesian SR method is proposed in this work. Different from the previous work, here, the high-resolution barcode image is estimated through its corresponding posterior probability distribution. The barcode image is made of some homogeneous regions separated by sharp edges, and sometimes it is anisotropic. A universal prior probability distribution was proposed for the barcode image by considering these characteristics. Mathematically, the efficiency of this prior distribution is demonstrated, which can preserve sharp edges and suppress artifacts in the reconstructed barcode images. Moreover, by using the variational Bayesian framework, the motion parameters and hyperparameters can be estimated accurately and efficiently, ensuring the success of the SR technique. In order to overcome the difficulty caused by nonlinearity, the Taylor expansion method is introduced to solve the proposed SR problem. Eventually, the simulated and real data experiments show the encouraging performance of the proposed SR method. It increases certainly the reconstruction quality, and could be considerably robust against blur and noise. It is believed that the variational SR technique in the barcode auto-identification technique should open a further perspective of coping with technology challenge.

In this paper, a fast background subtraction algorithm for freely moving cameras is presented. A nonparametric sample consensus model is employed as the appearance background model. The as-similar-as-possible warping technique, which obtains multiple homographies for different regions of the frame, is introduced to robustly estimate and compensate the camera motion between the consecutive frames. Unlike previous methods, our algorithm does not need any preprocess step for computing the dense optical flow or point trajectories. Instead, a superpixel-based seeded region growing scheme is proposed to extend the motion cue based on the sparse optical flow to the entire image. Then, a superpixel-based temporal coherent Markov random field optimization framework is built on the raw segmentations from the background model and the motion cue, and the final background/foreground labels are obtained using the graph-cut algorithm. Extensive experimental evaluations show that our algorithm achieves satisfactory accuracy, while being much faster than the state-of-the-art competing methods.

Numerous tracking-by-detection methods have been proposed for robust visual tracking, among which compressive tracking (CT) has obtained some promising results. A scale-adaptive CT method based on multifeature integration is presented to improve the robustness and accuracy of CT. We introduce a keypoint-based model to achieve the accurate scale estimation, which can additionally give a prior location of the target. Furthermore, by the high efficiency of data-independent random projection matrix, multiple features are integrated into an effective appearance model to construct the naïve Bayes classifier. At last, an adaptive update scheme is proposed to update the classifier conservatively. Experiments on various challenging sequences demonstrate substantial improvements by our proposed tracker over CT and other state-of-the-art trackers in terms of dealing with scale variation, abrupt motion, deformation, and illumination changes.

Most state-of-the-art single image blind deblurring techniques are still sensitive to image noise, leading to serious performance degradation in their blur kernel estimation when the input image noise increases. We found that reliable kernel estimation could not be given by directly using denoising and existing deblurring algorithms in many cases. We focus on how to estimate a good blur kernel from a noisy blurred image via using the image structure. First, we applied denoising as a preprocess to remove the input image noise and then computed salient image structure of the denoised result based on the total variation model. We also applied a gradient selection method to remove those salient edges that have a possible adverse effect on blur kernel estimation. Next, we adopted a two-phase estimation strategy to obtain higher quality blur kernel estimation by jointly applying kernel estimation from salient image structure and iterative support detection (ISD) kernel refinement. Finally, we used the nonblind deconvolution method based on sparse prior knowledge to restore the latent image. Extensive experiments testify to the superiority of the proposed method over state-of-the-art algorithms, both qualitatively and quantitatively.

The three-dimensional high-efficiency video coding (3-D-HEVC) is an emerging compression standard for multiview video plus depth data. In addition to the quad-tree coding structure inherited from HEVC, some tools are integrated, which significantly improve the coding efficiency but also result in remarkably high computational complexity. We propose a fast coding unit (CU) size decision algorithm for both depth and texture components in dependent views, where hole-filling maps created through view synthesis are utilized. First, after coding the base view, warp it onto each dependent view via depth image based rendering, during which hole-filling maps are generated. Then for depth in dependent views, CU splitting can be early terminated considering the disoccluded information from hole-filling maps; for texture in dependent views, combining the disoccluded information and the interview correlations, the CU partitioning process can also be accelerated. Experimental results show that the proposed algorithm can achieve on average 54.3% time reduction, with a negligible Bjøntegaard delta bitrate increase of 0.15% on synthesized views, and a 0.05% increase on all the coded plus synthesized views compared with the original encoding scheme in a 3-D-HEVC test model.

Using video in traffic monitoring is one of the most active research domains in the computer vision community. TrafficMonitor, a system that employs a hybrid approach for automatic vehicle tracking and classification on highways using a simple stationary calibrated camera, is presented. The proposed system consists of three modules: vehicle detection, vehicle tracking, and vehicle classification. Moving vehicles are detected by an enhanced Gaussian mixture model background estimation algorithm. The design includes a technique to resolve the occlusion problem by using a combination of two-dimensional proximity tracking algorithm and the Kanade–Lucas–Tomasi feature tracking algorithm. The last module classifies the shapes identified into five vehicle categories: motorcycle, car, van, bus, and truck by using three-dimensional templates and an algorithm based on histogram of oriented gradients and the support vector machine classifier. Several experiments have been performed using both real and simulated traffic in order to validate the system. The experiments were conducted on GRAM-RTM dataset and a proper real video dataset which is made publicly available as part of this work.

The purpose of image compression is to store or transmit image data efficiently. However, most compression methods emphasize the compression ratio rather than the throughput. We propose an encoding process and rules, and consequently a fast-adaptive near-lossless image compression method (FAIC) with good compression ratio. FAIC is a single-pass method, which removes bits from each codeword, then predicts the next pixel value through localized edge detection techniques, and finally uses Golomb–Rice codes to encode the residuals. FAIC uses only logical operations, bitwise operations, additions, and subtractions. Meanwhile, it eliminates the slow operations (e.g., multiplication, division, and logarithm) and the complex entropy coder, which can be a bottleneck in hardware implementations. Besides, FAIC does not depend on any precomputed tables or parameters. Experimental results demonstrate that FAIC achieves good balance between compression ratio and computational complexity in certain range (e.g., peak signal-to-noise ratio <35  dB, bits per pixel<2). It is suitable for applications in which the amount of data is huge or the computation power is limited.

The three-dimensional (3-D) video extension of high-efficiency video coding is an emerging coding standard for multiple-view-plus-depth that allows view synthesis for multiple displays with depth information. In order to avoid mixing between the foreground and background, the depth discontinuities defined at the object boundary should be retained. To solve this issue, a depth intramode, i.e., the depth-modeling mode (DMM), is introduced in 3-D-high-efficiency video coding as an edge predictor. The test model HTM 8.1 includes DMM1 and DMM3. However, the mode-decision strategy of DMM increases the complexity drastically. Therefore, we propose a fast DMM1 decision algorithm that estimates sharp edges by a subregional search method. The optimal wedgelet pattern of DMM1 is then searched only in the most probable region. Additionally, another fast method is raised to skip DMM3 when mismatch occurs between the depth prediction unit (PU) and its colocated texture PU. Simulation results show that the proposed algorithm has slightly better performance in terms of complexity reduction compared with the wedgelet-pattern-reducing algorithm from the literature while better maintaining the coding performance. In addition, the proposed algorithm has a performance similar to that of an existing DMM-skipping algorithm. Moreover, it could be integrated with that category of algorithms for additional time savings.

High-dynamic-range (HDR) imaging is becoming increasingly popular and widespread. The most common multishot HDR approach, based on multiple low-dynamic-range images captured with different exposures, has difficulties in handling camera and object movements. The spatially varying exposures (SVE) technology provides a solution to overcome this limitation by obtaining multiple exposures of the scene in only one shot but suffers from a loss in spatial resolution of the captured image. While aperiodic assignment of exposures has been shown to be advantageous during reconstruction in alleviating resolution loss, almost all the existing imaging sensors use the square pixel layout, which is a periodic tiling of square pixels. We propose the Penrose pixel layout, using pixels in aperiodic rhombus Penrose tiling, for HDR imaging. With the SVE technology, Penrose pixel layout has both exposure and pixel aperiodicities. To investigate its performance, we have to reconstruct HDR images in square pixel layout from Penrose raw images with SVE. Since the two pixel layouts are different, the traditional HDR reconstruction methods are not applicable. We develop a reconstruction method for Penrose pixel layout using a Gaussian mixture model for regularization. Both quantitative and qualitative results show the superiority of Penrose pixel layout over square pixel layout.

Optimization of the measurement matrix is one of the important research aspects of compressive sensing theory. A measurement matrix optimization method is presented based on the orthogonal similarity transformation of the information operator’s Gram matrix. In terms of the fact that the information operator’s Gram matrix is a singular symmetric matrix, a simplified orthogonal similarity transformation is deduced, and thus the simplified diagonal matrix that is orthogonally similar to it is obtained. Then an approximation of the Gram matrix is obtained by letting all the nonzero diagonal entries of the simplified diagonal matrix equal their average value. Thus an optimized measurement matrix can be acquired according to its relationship with the information operator. Results of experiments show that the optimized measurement matrix compared to the random measurement matrix is less coherent with dictionaries. The relative signal recovery error also declines when the proposed measurement matrix is utilized.

We formulate image segmentation as subspace clustering of image feature vectors. We propose a subspace representation model using a nonconvex extension of trace Lasso and a nonconvex approximation of rank function to regularize the subspace representation. The proposed model can adaptively capture the local and the global structures of the subspace representation so that the subspace representation can reveal the real subspace structure of the data and obtains excellent clustering performance. Experimental results show that the proposed model is better than the previous models in clustering and natural image segmentation.