JEI thanks the reviewers who served the journal in 2022.

A fingerprint identification system is an application of pattern recognition and image processing. The performance of a fingerprint-based biometric system relies on pre-processing techniques employed on fingerprint images. Especially, thresholding and thinning methods are used to detect minutiae points representing local features and are often utilized to identify a person uniquely. However, studies on partial fingerprints exposed the MasterPrint vulnerability for partial fingerprint identification systems wherein the system performs non-unique user identification. The thresholding and thinning techniques may lead to spurious minutiae generation and stimulate huge MasterPrints. Here, we investigate the impact of thresholding and thinning methods on identification accuracy and the percentage of MasterPrint generated using a partial fingerprint identification method. The experiments comprise four thresholding methods, namely, iterative optimal thresholding, Otsu’s global image thresholding, Niblack local thresholding, and Bernsen’s local image thresholding. Furthermore, it employs four thinning methods, namely, Khalid, Mariusz, Marek thinning algorithm, Khalid, Marek, Mariusz, Marcin thinning algorithm, Hilditch thinning algorithm, and Stentiford thinning algorithm. The results demonstrate that the identification accuracy and percentage of MasterPrint generated varies significantly by replacing the underlying pre-processing methods. Consequently, each combination of thresholding and thinning methods might not be suitable for user identification in high-security applications using a partial fingerprint identification method. The investigation outcomes provide the guidelines to demonstrate the robustness of a partial fingerprint identification method.

A highly efficient circle positioning algorithm, called the two-step optimization Hough transform (TSHT), based on multi-resolution segmentation is proposed to solve the problems of the offset Hough transform, namely, its large memory overhead, long time consumption, and low recognition accuracy. First, using the image feature of the printed circuit board (PCB) circular identifier, the target circle is obtained using adaptive image preprocessing, and then, images of an acceptable quality are separated by shape quality inspection to improve their robustness. Second, using effective interval sampling strategies and gradually controlling the accumulative interval of parameters, the TSHT algorithm reduces the memory overhead and quickly locates the center at the pixel level. Finally, the center at the sub-pixel level is found by the least-squares method for circle fitting. The experiments prove that TSHT, as a result of its high robustness, strong anti-noise capability, fast recognition speed, and accuracy, can be successfully applied to a vision positioning system of a solder paste printing machine.

Soccer player and ball detection and tracking have emerged as an area of intense interest among many analysts and researchers. This is because it aids coaches in team performance evaluation and decision-making to achieve optimal results. However, existing methodologies have failed to effectively detect and track the ball when it moves at high velocity and also to track players under occlusion conditions. You only look once (YOLOv3) and simple online real-time (SORT)-based soccer ball and player tracking approach is proposed, for accurately classifying the detected objects in soccer video and track them in various challenging situations. The proposed methodology consists of two parts: (i) YOLOv3 can detect and classify the objects (i.e., player, soccer ball, and background) and eliminate the detected objects outside the playfield as background; (ii) tracking is achieved using SORT algorithm which employs a Kalman filtering and bounding box overlap. The proposed model achieves tracking accuracy of 93.7% on multiple object tracking accuracy metrics with a detection speed of 23.7 frames per second (FPS) and a tracking speed of 11.3 FPS.

Monocular three-dimensional (3D) object detection (OD) is an essential and challenging task in the domain of autonomous driving. Modern convolution neural network-based architectures for OD heavily rely on data augmentation (DA) and self-supervised learning (SSL). However, they have been relatively less explored for monocular 3D OD, especially in the field of autonomous driving. DAs for two-dimensional OD techniques do not directly extend to the 3D objects. Literature shows that this requires adaptation of the 3D geometry of the input scene and synthesis of new viewpoints. This requires accurate depth information of the scene which may not be available always. We propose augmentations for monocular 3D OD without creating view synthesis. The proposed method uses DA with SSL approach via multiobject labeling as the pretext task. We evaluate the proposed DA-SSL approach on RTM3D detection model (baseline), with and without the application of DA. The results demonstrate improvements between 2% and 3% in mAP 3D and 0.9% to 1.5% BEV scores using SSL over the baseline scores. We propose an inverse class frequency weighted (ICFW) mAP score that highlights improvements in detection for low-frequency classes in a class imbalanced datasets with long tails. We observe improvements in both ICFW mAP 3D and Bird’s Eye View (BEV) scores to take into account the class imbalance in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) validation dataset. We achieve 4% to 5% increase in ICFW metrics with the pretext task.

Hyperspectral imagery gives details of spectral information through hundreds of spectral bands also known as dimensionality. The bands with continuous spectral information model are capable of classifying various materials of interest. The enhanced dimensionality of such data allows for major changes in data relevant information, but it also presents a challenge to traditional methods for accurate hyperspectral image analysis so-called “curse of dimensionality.” The hyperspectral images are used to identify objects on the earth’s surface. Due to a large number of spectral bands, classifying objects using hyperspectral imagery has become more challenging. Feature reduction and extraction techniques are used to address these high-dimensionality issues. However, there are various challenges dealing with data classification with performance and computational time. As a result, a technique for hyperspectral image classification based on a convolutional neural network (CNN) along with geometric transformation and polygons segmentation process has been proposed. A CNN is used to convert compressed features into high-level features that were utilized to classify objects into buildings and road networks. The main objective of this paper is to design an automated building footprint extraction and road detection from hyperspectral imagery using CNN. The polygons segmentation is used for the extraction and detection of spectral features in hyperspectral data. CNN is used to classify these extracted spectral features, such as building footprints and road detection, using different kernels. When comparing the proposed techniques with other support vector machine and fully convolutional network methods, the experimental results show that CNN provides 97% classification accuracy.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of compromised peer review and determined the paper has little relevance to the scope of the journal. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JS disagrees with the retraction. SNK, KB, FR, TY, and AJ either did not respond or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

HY and ZL either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

XZ either did not respond directly or could not be reached.

Since the 1980s, various art and cultural activities from the West have been widely disseminated in our country, and sports dance is one of the most popular projects among them. However, the training for sports dance mostly stays in the traditional classroom, and is not combined with deep-learning techniques. We focus on the video analysis and intelligent diagnosis system of sports dance training projects. During the research process, we developed an intelligent platform dedicated to training projects and combined it with the signal processing diagnostic system. The effect of training will ultimately lead to better competition results. From the experimental data, in this test of ten dance styles, the highest comprehensive score is Waltz’s 8.93 points, and the lowest comprehensive score is Samba’s 8.312 points. Moreover, the system designed is superior to the traditional multimedia system in performance comparison, which can provide better help for sports dance training. On the whole, the application of the intelligent diagnosis system developed in sports dance training projects meets the expected demand.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JZ either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

QJ, YW, and PG either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

LS did not agree with the retraction. YM, YL, and LC either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

SM and ZG did not agree with the retraction.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

XY, XZ, WL, XL, and PS did not agree with the retraction.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

QL and QH did not agree with the retraction. FZ and ZL either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

DKJ, XL, SN, and MP either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

ZM did not agree with the retraction.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

LS did not agree with the retraction. ZW, NC, and JC either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

XZ either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

Pin Wang agrees with the retraction. Peng Wang does not agree with the retraction. ZG, LZ, and HZ either did not respond or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

ZS does not agree with the retraction. CF, YL, and HZ either did not respond or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

RK did not agree with the retraction. HL, CJ, and MS either did not respond directly or could not be reached.

Building vector extraction from aerial images is a challenge in many applications, especially location-based services. In recent years, different deep-learning techniques have improved the accuracy of building extraction. We propose a framework (JuliMa-Net) for building vector extraction from aerial images without boundary regularization or vertex sampling. The wireframe generated by our framework shows more detail about the building’s structure than its footprints. We initially selected three pretrained networks of Mask R-convolutional neural network, line, and junction detection. To improve the performance of the primary junction detection network, unique sets of decoders were designed. The smaller training set was used to fine-tune all three networks simultaneously. In addition, after applying proper processing on the obtained building masks, the outputs of the other two networks—the detected lines and junctions—were precisely selected and combined. The result of this process is a significant reduction in false detections and an increase in precision of 96%. Additionally, the final processing adds precise lines to the wireframes after combining junctions and lines. This improves the recall value.

Model-based methods utilize atmospheric scattering model to effectively dehaze images but introduce unwanted artifacts. By contrast, recent model-free methods directly restore dehazed images by an end-to-end network and avoid artificial errors. However, their dehazing ability is limited. To address this problem, we combine the advantages of supervised and unsupervised learning and propose a semisupervised knowledge distillation network for single image dehazing named SSKDN. Specially, we respectively build a supervised learning branch and an unsupervised learning branch by four attention-guided feature extraction blocks. In the supervised learning branch, the network is optimized by synthetic images. In the unsupervised learning branch, we dehaze real-world images by dark channel prior and refine dehazing network (RefineDNet) (another dehazing method) and use these dehazed images as fake ground truths to optimize network using prior information and knowledge distillation. Experimental results on synthetic and real-world images demonstrate that the proposed SSKDN performs better than state-of-the-art methods and owns powerful generalization ability.

The conventional image compression framework is pixel fidelity-driven, which can generate compressed images with considerable visual quality even at low bit rates. However, these methods emphasize the human visual experience and ignore the need for machine recognition-driven tasks. To this end, we propose an image compression framework that utilizes multiscale prior information extracted from the machine perceptual model to improve the machine recognition accuracy of compressed images. Specifically, the interaction refinement module (IRM) is designed to interact multiscale prior information with each other, adaptively retaining machine recognition–relevant features to enhance its expression on compact features. To further improve the accuracy of machine recognition, machine vision perceptual loss is designed on semantic variation weight, which is the weight of semantic variation degree of deep adjacent layers in multiscale priors. Machine vision perceptual loss is used to optimize the semantic distortion of compressed images for retaining important semantic information. Experimental results show that compared with compression methods including BPG, WebP, Mentzer, NIC, IUWD, and RCIS, the Top-1 recognition accuracy of the proposed method is improved by 10.9%, 19%, 11.6%, 12.9%, 6%, and 2.7% at a lower bit rate (0.2 bpp). In addition, the performance improvement on other machine recognition networks and machine vision tasks shows the versatility of the proposed method.

An image denoising method is proposed for ultrasonic logging images with severe noise. The proposed method works on a variational Bayesian framework using block sparse prior. First, the sparse coefficients are simulated by a more appropriate distribution—Laplacian distribution. Then the variational Bayesian denoising model in which Laplacian distribution is used as a prior term of sparse coefficients is proposed. Finally, semiquadratic regularization is used to solve the model with a simplified process. Moreover, during the denoising process, a relaxation factor is introduced to improve the accuracy. In the vast majority of cases, the proposed method obtained better results in both the visual quality and the objective evaluation. It achieves better denoising performance than the existing denoising methods when the edge details of the images are contaminated by noise, especially severe noise. The experimental results show that the proposed method is practical in ultrasonic logging images.

Free-hand sketches play a vital role in graphically portraying ideas and concepts in image recognition systems. Most recently proposed learning-based sketch recognition methods have achieved marked progress in recognition accuracy, but they rarely optimize the use of the sparsity features of sketch images. Although several attention-based sketch recognition models have been presented, they endure complex computations and large model sizes. To address these challenges, we present a lightweight convolutional neural network called Light-SRNet based on a dual-attention mechanism to improve the accuracy of sketch recognition while retaining its lightweight nature. In the proposed model, we introduced both the spatial and channel attention mechanisms into the feature extraction network to highlight more discriminative feature representations to enhance its powerful sketch recognition ability. We compared the proposed Light-SRNet with its competitors on the TU-Berlin dataset, Sketchy dataset, and QuickDrawExtended dataset. Extensive experimental results show that Light-SRNet achieves a recognition accuracy of 73.14%, which is comparable to other similar sketch recognition techniques, while requiring only about a quarter of the model parameters.

Denoising convolutional neural network (DnCNN) has achieved competitive denoising performance for Gaussian noise using residual learning. The same idea can also be applied in the impulse noise (IN) removal by changing the training data. However, as a discriminative learning-based algorithm, DnCNN lacks flexibility in removing spatially varying noise, such as random-valued impulse noise (RVIN). To address this problem, we propose a fine-tuning CNN based on relaxed Bayesian-optimized support vector machine (BO-SVM) for RVIN removal using a relaxed BO-SVM as a detector and a fine-tuning DnCNN as a denoiser. The relaxed BO-SVM is developed to classify all pixels of the test image into two categories: noise and noise-free. A relaxed rule is provided to preserve the information of slightly corrupted pixels, and the system is trained with an optimal feature set, which comprises of newly introduced local ternary pattern with previously used gray-level difference, average background difference, and median value difference. The key hyperparameters of the relaxed SVM are obtained using the Bayesian optimization technique. Then the DnCNN works for image reconstruction through fine-tuning experiments by the use of the corrupted pixels identified by the detector. The experimental results show that the proposed method reports promising results on RVIN removal over other state-of-the-art denoising methods.

In real life, the emergence of haze brings great inconvenience and does harm to traffic and pedestrian safety, whereas previous studies paid less attention to text detection in haze scenes. In this experiment, we found that the candidates obtained by the general non-maximum suppression (NMS) method or the soft-NMS method are difficult to precisely match the ground truth, and the incomplete feature extraction will affect the final performance. In this work, a haze scene text detection framework is skillfully designed. An optimized NMS and an optimized long short-term memory for spatial feature extraction and temporal feature extraction are proposed to improve the text detection performance. In addition, a hazing scene text dataset (named HSText-1000) and a hybrid scenario text dataset (named MHSText-4600) have been built in our work for evaluating the performance conveniently, which have been released and can be downloaded from https://github.com/lyy0117/lyy. Experimental results illustrate that our method is superior to some state-of-the-art methods in the hazing scene and the hybrid scene. Meanwhile, we achieved competitive results in nonhaze’s public dataset (ICDAR 2013), which means that our method has satisfactory adaptability. We will release code to facilitate community research.

Retinal biometric trait is one of the safest and most secure biometric traits that can be used for personal identification and authentication. The poor contrast in captured retinal images initially requires the adoption of a preprocessing technique, especially when the vessel segmentation is required. As a result, an effective preprocessing technique is employed before the feature extraction process. The reliability of the segmentation depends on the consistency of the contrast across the images. Due to the complexity of the retinal vascular patterns, segmenting the vessels manually is a time-consuming task that also requires extensive training and abilities to distinguish the thinner blood vessel pixels from the background of the retinal images. We present a unique blood vessel segmentation technique based on Radial Chebyshev Moments (RCMs) shape descriptors. Moment-based shape descriptors are typically strong, concise, and simple to compute. They are also insensitive to the object’s scale, rotation, and translation. Furthermore, to eliminate the eye movement and to overcome the various lighting problems, the following multiple features are extracted in the feature extraction stage to accurately authenticate a person: statistical (gray-level) features, shape-based (RCM) features, and structural features, such as minutiae, bifurcation points, optical disc, and enclosed regions. The principal components analysis methodology is used to optimize the feature vector, and the target image is identified using the support vector machine classifier. This step helps to determine whether the test image belongs to the authorized person or not. RIDB and VARIA are the two standard retinal authentication databases and DRIVE and STARE are the two general retinal databases that are used for evaluating the performance of the system. The system yielded a high identification rate of 98%, 97.5%, 97.2% and 94.4% with average processing time of 0.987 s, 0.40 s, 1.262 s and 0.65 s, respectively. From the results, it is evident that the proposed method is more effective and efficient for authenticating individuals.

To apply the quaternion moments to the image feature extraction process and improve the performance of the image hashing algorithm, we propose a hashing algorithm based on quaternion polar complex exponential transform and image energy. First, the input color image is preprocessed to form a secondary image. The secondary image is subjected to fast quaternion generic polar complex exponential transform to obtain the mixed low-order moment coefficients as the global features of the image. Second, the algorithm extracts energy change features on the edges of the secondary image and energy point features in the center of the image as energy features of the image under the YCbCr color space. Finally, the image’s mixed low-order moment and energy features are combined and scrambled to form the final hash sequence. The experimental results show that the proposed algorithm balances robustness and differentiation well. Moreover, the hash sequence is the shortest, and the algorithm has better check-all and check-accuracy rates compared with other algorithms in the copy detection experiments.

Point cloud registration is an important research problem in machine vision. One of the most widely used methods for point set registration is the iterative closest point (ICP) algorithm. However, ICP is known to be susceptible to local minima because it is based on the local iterative optimization technique and its performance critically relies on the quality of the initialization. To reduce the possibility of the algorithm falling into local minima, we propose an iterative algorithm based on geometric distance and local feature weighting named weighted correlation coefficient iterative closest point algorithm (WCC_ICP). The algorithm first establishes the corresponding relationship of points using linear representation and constructs an optimization model with constraints. Subsequently, the constrained optimization model is converted into an unconstrained optimization model by Lagrangian multiplication. Finally, an iterative technique is used to solve the optimal rigid body transformation of the optimized model. In addition, in each iteration, the identification ability of the local feature operator of points is used to repair the corresponding relation of points, which reduces the risk of the iterative algorithm falling into the local minima. Numerical experiments show that the proposed WCC_ICP algorithm reduces the risk of the iterative algorithm falling into local minima.

In recent years, skeleton-based action recognition has become increasingly popular in the field of human action recognition, and graph convolutional networks (GCNs) have shown better advantages in this task. Many GCN-based methods are insufficient in the latent relationship between features, which affects the discriminability of features being not rich enough. These potential feature relationships can manifest as feature differences that change due to actions and feature correlations that interact with each other. Therefore, we propose a feature difference and feature correlation learning mechanism to learn discriminative augmentation features, including feature differences in actions and feature correlations between joints. First, we propose a temporal feature difference and correlation learning module (FDCL) (TFDCL). In adjacent temporal frames, we extract feature correlations between related parts. Feature differences are captured through changes in joints over the overall long-term timeline. Second, we propose a channel FDCL module. Different channels contain different types of features for actions. We use convolution operations to interact between channels, continuously extracting the strongest features to obtain feature maps. Third, we propose a temporal channel context topology (TCCT) module to dynamically learn global contextual features of all joints during motion. Finally, experiments are conducted on the NTU-RGBD 60 dataset and the kinetics-skeleton 400 dataset to verify the effectiveness of the network.

Internet of Things (IoT) connects a wide range of small devices over a large network, allowing for a wide range of applications. With the advancement of open-source public networks (such as IoT), image transmission for resource-constrained devices is becoming more insecure. In this paper, two hardware implementations for KLEIN block cipher are proposed that can provide security to encrypt different sets of images under resource-constrained applications (such as wireless sensor network, radio frequency identification, IoT etc.). An improvement in maximum operating frequency of 32.72% and 75.66% in the proposed serial and pipelined architectures is observed compared with the state-of-the-art design, respectively. The implemented pipelined architecture yields high throughput of 1840.15 MHz, and the serialized architecture implies 108 numbers of slices implemented on field-programmable gate array (FPGA) xc5vlx50t-3ff1136 device. On application-specific integrated circuit (ASIC) platform, the proposed serialized KLEIN implementation consumed less area in terms of gate equivalent. The existing solution was also subjected to changing correlation coefficients, number of pixels changing rate, unified average changing intensity, and entropy values. The simulation results validate the effectiveness of the performance of the proposed encryption scheme.

With the wide popularity of surveillance cameras, video synopsis technology has become a research hotspot. The existing methods of surveillance video synopsis usually summarize the input video by shifting the object tube in the video on the time axis, which ignore the serious collision artifacts and chronological disorder between moving objects. To solve these problems, we propose a surveillance video synopsis methodology called “surveillance video synopsis based on spatio-temporal offset (STO)” that can simultaneously shift the moving object in the temporal domain and spatial domain. First, object detection and tracking algorithms are used to extract the object tube from the input video. Two collision relations are proposed by analyzing relationship between tubes to classify collision artifacts. Then, we present two spatial offset cases to find the optimal spatial offset of the object tube. Finally, an adaptive optimization frame density model is proposed to analyze the optimal temporal offset of the object tube. Simultaneously, the object tube and the background are stitched according to the STO to generate the synopsis video. Extensive experimental results demonstrate the effectiveness of the proposed method in improving frame compression rate and alleviating collision artifacts.

Due to the limited computing power of unmanned aerial vehicles (UAVs) and the problems of missed detection and wrong detection of small objects, the current object detection algorithm cannot achieve real-time and high-precision detection. To solve these problems, we propose a vehicle detection network Shuffle CarNet for UAVs aerial images, which is composed of a feature extraction network, a feature fusion network, and a three-scale prediction network. First, according to the limited hardware resources of embedded devices, a lightweight feature extraction network Light CarNet is proposed by fusing the attention mechanism. Second, a four-scale feature bidirectional weighted fusion module is designed. According to the characteristics of the object scale, multilevel feature map bidirectional weighted fusion is selected for target classification and bounding box regression on three scales. Finally, Car-non-maximum suppression is used to reduce false detection and missed detection. Experiments show that compared with other algorithms on the VisDrone-2019 dataset, the proposed method improves the mean average precision by 1.14%, achieves a precision of 82.96%, and can meet the needs of real-time vehicle detection. The superiority of this method is proved by many comparative experiments.

Face-spoofing detection plays an important role in ensuring the security of face recognition systems. Most multi-modal methods based on deep learning improve their accuracy by utilizing information from RGB, depth, and infrared. In fact, given the cost and application conditions, it is difficult to obtain all these data. Therefore, it is especially important to exploit single-modal images to extract more detailed information. To address the above problems, we propose an efficient two-stream convolutional network, which takes an original image and its wavelet-transformed image as input. Then, we design two branches to extract the features, with the wavelet branch more conducive to mining the detailed information. Finally, we adopt three loss functions to supervise the two branches and the fused branch respectively, and each branch can be scored separately. The extensive experiments demonstrate that our model can achieve satisfactory performance on the datasets, with replay-attack and CASIA-FASD achieving 100% accuracy.

A long-standing problem in melanoma segmentation is that the pixel-level skin lesion label must be manually labeled by experts. Semi-supervised learning is considered a promising way to solve this problem. However, the pseudo labels obtained in the training phase are not accurate and will introduce wrong guidance to the model’s training. The segmentation of the unlabeled data after filtering by the confidence map can be a reliable pseudo label in training processing. Thus, a confidence-aware cross-supervised network is proposed for skin lesion segmentation, using captured confidence to make the training process more reliable. Our method is composed of two parallel and independently initialized segmentation networks. The output of one segmentation branch filters the low-confidence area by the confidence map of the same network and is used as a pseudo label to supervise another branch. Furthermore, a consistency loss is designed to calculate cross-entropy between the pseudo label and prediction segmentation to make full use of the unlabeled data. We also propose a simple yet effective attention module to capture the feature from the target area. This approach effectively mines the semantic information of the input image. To validate the effectiveness of our method, we conduct experiments on the ISIC 2017 dataset and achieve state-of-the-art results.

Retail product detection in fisheye camera capture scenes frequently suffers from excessive object occlusion and deformation, as well as difficulty in distinguishing products with small fine-grained differences, so accurately classifying and localizing products in these images presents a challenge for computer vision. We propose an efficient product detection network called EPformer by fusing a visual transformer and convolutional neural network to reliably detect retail products in fisheye images. We employ a shifted window strategy for feature information interaction across windows to more precisely detect products due to the issue of dense occlusion of products. To address the issue of excessive product deformation brought on by fisheye cameras, we develop a deformation image processing module without explicit correction and embed it into the path aggregation network structure. This enables the model to efficiently capture product geometric changes and conduct feature fusion. To address the issue of differentiating fine-grained products, we design an effective coordinate squeeze-excitation (ECSE) attention module that can capture the fine-grained texture and boundary information differences between individuals in terms of spatial and channel relationships. The inability to differentiate fine-grained products can be solved by training the ECSE module in tandem with the decoupled head. The experimental results demonstrate that EPformer is a potent product detection model with a 4.9% higher mean average precision than the state-of-the-art method (YOLOX) on the fisheye product image dataset. In addition, the EPformer model can effectively detect products in fisheye images on the Jeston Xavier NX embedded device to meet the application requirements in realistic scenarios.

Object detection and analysis in remote sensing images is a critical research subject for many businesses and agencies. At present, object detection based on convolutional neural network (CNN) in natural scenes has good performance. Due to the large number of small objects and similar characteristics between the objects in the VisDrone dataset, the current model cannot extract more small-scale features. Therefore, this paper proposes a stronger feature extraction FasterRCNN (SFE-FasterRCNN) that advances a feature extraction strengthening network to enhance the feature learning ability for different objects. Specifically, the pixel proposal network (PPN) is proposed by combining the low-resolution and strong semantic features with high-resolution and weak semantic features through a top-down approach and reusing these fusion blocks vertically to construct a comprehensive semantic feature map. Then hyperbolic pooling is proposed to minimize the loss of feature information during the activation mapping process. Finally, data clustering is used to adaptively generate better object proposals according to the characteristics of the dataset. Experimental results on the VisDrone dataset show that our method has excellent detection results.

The local regions of complex gyration class mechanical parts are largely different, which results in the problem that their surface images are partially clear. This problem directly affects the effectiveness of vision methods on detecting surface defects. To tackle this problem, a large depth-of-field (DoF) surface image fusion method is proposed to obtain a full-focus image of complex mechanical parts, such as gyration class mechanical parts. We design an image acquisition platform based on the characteristics of gyration class parts to acquire their surface images accurately and clearly. We then propose a feature detection-based image registration method, by which the registered image can represent the surface information of the part completely and accurately. Additionally, we adopt a convolutional neural network-based image fusion method to achieve a fused large DoF surface image. Experimental studies were conducted to evaluate the performance of the deep-learning-based methods. The experimental results show that the proposed method can completely and clearly fuse the large DoF surface images of complex gyration class mechanical parts. The quality of the fused images has a significant improvement, and the proposed method is significantly more efficient than traditional fusion methods and other deep-learning methods.

Through comparison with most frequency-domain digital watermarking approaches, the spread transform dither modulation (STDM) method exhibits a good embedding effect and is capable of meeting the requirements of blind extraction. However, the rotating attack is difficult to counter, and the cropping attack is seriously impacted by location factors. In addition, there exist numerous algorithms concentrate solely on the watermarking process, ignoring the image’s security. In these views, our work proposes a two-dimensional hyperchaotic system, the so-named Chebyshev ICMIC cascade map with the Knuth–Durstenfeld method, aiming to enhance the security of digital watermarking systems. The scrambled pixels improve the anticropping attack effect through breaking the location correlation of watermark information. For the purpose of magnifying resistance to rotation attack impact, the radial harmonic Fourier moments subtraction coefficient matrix distribution expectation is appropriately adapted to carry out the rotation correction. The performed experiments reveal that the suggested improved STDM watermarking algorithm demonstrates better robustness and invisibility, resisting more attacks than the existing methodologies. Finally, the feasibility of the proposed methods is proved based on the simulation results.

Recently, an unsupervised method [denoted as deep image prior (DIP)] that interprets images as the output of a convolutional network with randomly initialized inputs has received much attention, as the DIP can adapt its network parameters to any given noisy image without requiring training pairs, providing a great flexibility for handling those noisy images with complex structures and low degree of self-similarity. However, the DIP method still suffers from a crucial limitation that only the noisy image is used as target image, which implies that only internal prior information is used for noise removal and leads to poor denoising performance, compared with supervised denoising models. In this work, we aim to boost the DIP method by integrating external and internal image priors. Specifically, with a given noisy image, we first exploited a state-of-the-art supervised method to denoise it and obtain its corresponding denoised image called initial denoised image. Then the initial denoised image containing rich external prior information was used as extra target image together with the given noisy image within the standard DIP framework, resulting in a highly effective unsupervised recovery process. Next, to take full advantage of the uncertainty of the DIP network, the above DIP denoising routine with the different random inputs was executed multiple times to generate enough complementary denoised images (samples). Finally, with an unsupervised weight map generative network, the generated samples were fused in a pixel-wise manner. The fused image with better image quality was treated as the final denoised image. We had verified the denoising performance of our method on a large of images from several benchmark databases as well as real-world noisy images. The comparative results show that our method outperforms the original DIP method and other unsupervised methods by a large margin, and surpasses state-of-the-art supervised counterparts with comparable peak signal-to-noise rate. Source code is publicly available at https://github.com/chenxiaojun0101/BDIP.

Most digital cameras have spectral characteristics that make it impossible for the camera to provide accurate colorimetric information of a recorded scene. We extend previous work on using multiple filtered exposures by the use of a newly available capability of creating manufacturable filters using online software. Multiple criteria are considered in the determination of an optimal color filter on a training dataset. These criteria include the Vora-Value, figure of merit (FOM), training average ΔE, and training maximum ΔE. Our method includes the use of a realistic imaging noise model. We use two optimization methods: minimum mean square error (MMSE), which requires knowledge of the color sensitivities of the camera’s internal RGB filters and knowledge of the covariance matrices of the data ensemble and the noise, and a pseudoinverse method that uses only the recorded data and knowledge of the values of the training data. Filters were selected using the training data and the performance is measured on independent testing data. The results from both optimization methods show that filters chosen using the training ΔE criteria consistently outperformed the theoretical FOMs. It is also shown that the MMSE method is very sensitive to small perturbations of the internal filter sensitivities. Real camera images were recorded using the optimally determined filters from both methods. Measured errors provide confirmation of the simulated results.

Discriminative correlation filter-based (DCF) object-tracking methods exhibit excellent and efficient performance in short-term tracking. However, these methods do not evaluate the credibility of the feature channels and spatial features. In addition, they regard the candidates with the largest response in each frame as the training sample without evaluating the confidence of these samples. As a result, DCF may be misled by distractors and fails in the occlusion scene. We propose a visual attention learning and anti-occlusion-based correlation filter tracking method to solve the abovementioned limitations. First, we propose a correlation filtering framework for simultaneous estimation of position and scale in which we sample the multi-scale detection candidates from the target’s center in the previous frame and calculate the spatial attention of each candidate according to the spatial prior and color likelihood. Then, we design a channel attention calculation algorithm to evaluate the importance of each channel according to the response in the learning and detection stages. Finally, a historical multi-template pool is proposed to further evaluate the confidence of the best candidate in each frame, achieving robust tracking in occlusion scenarios. A series of experiments are conducted to verify the proposed method, indicating that the proposed method achieves excellent results on the OTB50, OTB100, and UAV20L datasets and outperforms many state-of-the-art methods.

Image decolorization is used to realize the color-to-gray conversion, which is widely used in black-and-white printing, digital-ink display, photo stylization, etc. Image contrast fidelity preservation is the key issue during this conversion process. We present a parametric model combining image entropy and the Canny edge retention ratio (CERR). Maximizing the image entropy keeps the global image contrast content, while maximizing the CERR enhances the local image contrast, mainly along the edges. The weighting parameter λ can be robustly estimated. Experimental results on Cadik’s and the COLOR250 datasets demonstrate that the model can achieve significant performance improvement over state-of-the-art methods both qualitatively and quantitatively.

Multi-modal pedestrian detection, which integrates visible and thermal sensors, has been developed to overcome many limitations of visible-modal pedestrian detection, such as poor illumination, cluttered background, and occlusion. By adopting the combination of multiple modalities, we can efficiently detect pedestrians even with poor visibility. Nevertheless, the critical assumption of multi-modal pedestrian detection is that multi-modal images are perfectly aligned. In general, however, this assumption often becomes invalid in real-world situations. Viewpoints of the different modal sensors are usually different. Then, the positions of pedestrians on the different modal images have disparities. We proposed a multi-modal faster-RCNN specifically designed to handle misalignment between two modalities. The faster-RCNN consists of a region proposal network (RPN) and a detector. We introduce position regressors for both modalities in the RPN and the detector. Intersection over union (IoU) is one of the useful metrics for object detection but is defined only for a single-modal image. We extend it into multi-modal IoU to evaluate the preciseness of both modalities. Our experimental results with the proposed evaluation metrics demonstrate that the proposed method has comparable performance with state-of-the-art methods and outperforms them for data with significant misalignment.

This paper proposes a network model based on a three-stream network and improved attention mechanism for blind image quality assessment (TSAIQA). The inputs of the three streams are the distorted image, the pseudoreference image obtained by the improved generative adversarial network (GAN), and the gradient map of the distorted image. The distorted image stream focuses on the holistic quality-related features, the pseudoreference image stream is used to supplement the lost features due to distortion, and the gradient stream explicitly extracts the quality-related structural features. In addition, spatial and channel attention mechanisms combining first- and second-order information are proposed, and the improved attention mechanisms are applied to the three-stream network to optimize spatial and channel-level features effectively. Finally, the three-stream fusion features are input to the quality regression network to predict the image quality. To demonstrate the effectiveness of the proposed model, experiments are conducted on four classical IQA databases and two new large-scale databases. The experimental results show that the results of our TSAIQA model outperform the most advanced IQA methods and confirm the effectiveness of the proposed network structure and attention mechanisms.

The use of a biometric authentication system (BAS) for reliable automatic human recognition has increased exponentially over traditional authentication systems in recent years. Multimodal BAS was introduced to solve unimodal BAS’s difficulties and improve security. Privacy and security are two significant concerns to be addressed in BAS, as biometric traits are irrevocable. Researchers employed cancelable biometrics in the past few years to propose several privacy-preserving BAS. We propose a privacy-preserving bimodal cancelable BAS (PPBCBAS) to overcome these problems. The traits used in our method are iris and fingerprint. Features are extracted from both the traits, and feature level fusion is done by concatenating the feature vectors of iris and fingerprint. PPBCBAS uses a quotient filter to generate the cancelable template, and the comparison is made on these transformed templates using the modified Hamming distance. PPBCBAS has been tested on three publicly available databases to analyze its efficiency. PPBCBAS satisfies the diversity, irreversibility, and revocability properties and achieves decent performance.

Edge-preserving image smoothing is essential for computational imaging. Traditional filters are based on low-level image gradients; however, the edges defined on gradients do not align with the contours of human perception, thus suffering from certain limitations. We propose an image filter based on a soft clustering model that combines high-level semantics (derived from instance segmentation) with low-level features (intensities). The proposed filter works by first performing soft clustering on the input image according to the high- and low-level features to derive the affinity matrices, which are then fused for image smoothing. Experiment results indicate the advantages of the proposed filter in a variety of applications, including image smoothing, flash/non-flash fusion, detail enhancing, image dehazing, and depth upsampling. Furthermore, the proposed filter is efficient, and it takes 1.75 s to process a color image with 1 megapixel on a modern desktop.

The research of underwater sonar image classification is beneficial to the development of marine resources. In recent years, deep neural networks have achieved great success in the fields of image classification and image recognition. However, deep neural networks require a large amount of training data. Due to the difficulty of obtaining sonar image datasets, the existing public sonar datasets are generally small, and sonar image classification can be regarded as a small sample problem. To solve this problem, we designed a deep adaptive sonar image classification network (DASCN) based on deep learning and domain adaptation. The feature extraction module in DASCN extracts multiscale features of images; the attention module learns the importance of different channel features; and the domain adaptation module reduces the difference between the source domain and the target domain. It is worth noting that our DASCN does not require a large number of training samples, and sonar images used for training do not need to be labeled. As demonstrated in comprehensive experiments, the classification accuracy of the DASCN model reached 89.4% on the sonar image dataset. Our DASCN achieves unsupervised accurate classification of small sample sonar images. In addition, our DASCN has good classification results on the Office-31 dataset and the Office Home dataset and has good generalization performance.

The online multiobject tracking (MOT), which integrates object detection and tracking into a single network, has made breakthroughs in recent years. However, most online trackers have a more monotonous prediction of the tracking offset in two consecutive frames, which may not be reliable when facing extreme situations such as occlusion and object deformation. Once the tracking offset of an object is biased, its corresponding tracklet will no longer maintain temporal consistency, which will seriously affect the tracking performance. In this paper, we propose a new online multiple object tracker with feature enhancement mechanism, namely En-Tracker. In En-Tracker, a multibranch kinematic analysis network (MKANet) is designed to address the above problems. MKANet estimates the pixel offset and instance offset of the object in parallel based on imitating the human thinking to joint position and appearance representations. Note that these two types of offsets compensate and facilitate each other to effectively deal with some extreme scenarios. In addition, we propose a kinematic-assisted feature synthesis enhancement (KFSE) module, which has a more comprehensive enhancement mechanism. Specifically, KFSE propagates previous tracking information to the current frame based on kinematic trend analysis while enhancing the characterization of detection features and appearance embeddings, which not only assists in object detection but also ensures the uniqueness of the appearance embeddings. Extensive experiments on MOT16 and MOT17 verify the effectiveness and advantage of our model.

In the current state-of-the-art distributed video coding (DVC) solutions, side information (SI) frames are created by motion-compensated interpolation. This is the same technique used by frame rate upconversion operations. Recently, these operations are accomplished using deep learning (DL), with significantly improved outcomes. DL is also used to improve the decoded images of numerous compression methods. Consequently, it is natural to wonder what are the impacts of these enhancements on DVC. In this regard, we employed DL networks to create SI and enhance the quality of the DVC decoded frames. We show how these two operations were applied to DVC and their effects on rate-distortion (RD) performance. The comparison with state-of-the-art DVC systems shows an improvement in the quality of the SI frames and that of the decoded frames, and thus an improvement of the RD performance of DVC.

Here, we describe a new method for reversible data hiding in encrypted images (RDHEI) that aims to improve the embedding capacity (EC) and visual quality of the stego image. Most of the existing techniques for RDHEI need location maps to discriminate embedded and non-embedded pixels, which increases computational overhead and affects EC. The proposed method reserves room after encryption and converts prediction errors (PE) to odd and even numbers to discriminate between embedded and non-embedded pixels. This avoids the computational overhead caused by the use of a location map. The proposed scheme also employs double-layer encryption using random permutation and a stream cipher to secure the cover images. The method is separable, which means that the secret data and cover image can be independently extracted and recovered, depending on the availability of the data hiding and decryption keys. The scheme uses block-wise reference pixels for the calculation of PE, that enhances the EC. Experimental results indicate that the proposed scheme is more effective than existing state-of-the-art works.

Analysis dictionary learning (DL) has been successfully applied to the field of pattern classification. However, it is still a challenge to utilize the local structure information and the class information of samples to improve the discrimination capability of analysis dictionary. We proposed a joint structured constraint discriminant analysis DL (ADL) method (JSCDADL) to learn a structured discriminant analysis dictionary by combining the local structure information and the structured information of samples. Specifically, we first designed an adaptive local structure preserving term (ALSPT) to improve the discrimination capability of analysis dictionary. It adaptively transmits the local structure information of samples to analysis dictionary, which ensures that the same class of samples has similar sparse codes under the action of the analysis dictionary. Then, we designed a discriminative sparse coding error term that forces the coding coefficient matrix to have the desired block diagonal structure. To further enhance the discrimination capability of analysis dictionary, we designed an analysis dictionary combination term by constantly approximating the two analysis dictionaries learned to obtain an analysis dictionary with the local structure information and the structured information of samples. Moreover, we designed an effective iterative algorithm to solve the optimization problem of JSCDADL. Extensive experimental results on six datasets demonstrate that JSCDADL can achieve satisfactory classification performance compared with some state-of-the-art methods.

Digital camera sensors capture natural images in low-light conditions, resulting in poor imaging results. Existing low-light image enhancement (LLE) often yields unnatural results due to over-enhancement, artifacts, severe noise, etc. Prior studies either perform visual dual-pathway mechanisms or Retinex prior optimization for image enhancement. However, image enhancement based on the former generates artifacts because it directly stretches contrast in the structural layer with mixed high- and low-frequency information. The latter results in over-enhancement due to adding empirical prior items to the objective function. Thus, a unified three-pathway framework is proposed to address the aforementioned deficiencies for LLE. Specially, the proposed framework is composed of detail pathway, reflection pathway, and illuminance pathway. First, three information processing pathways can be obtained through different image decomposition strategies. Second, an indirect noise suppression strategy is developed in the computational flow of detail pathway and reflection pathway to address noise amplification problem of image enhancement. Third, the naturalness preservation enhancement task is conducted in the reflection pathway and illuminance pathway. Finally, the outputs of different pathways are weighted and fused to enhance low-light image. Moreover, qualitative and quantitative experimental results on two test datasets show that the proposed framework outperforms state-of-the-art methods.

In order to solve lane line detection in difficult traffic conditions, such as shadow occlusion, signpost degradation, curves, and tunnels, numerous models have been proposed. However, most of the existing models conduct independent single-frame image detection, which makes it difficult to utilize the continuity of driving images and is ineffective in challenging scenes. To this end, we suggest a spatiotemporal information processing model for lane line recognition that enhances critical features. In order to properly learn the correlation between continuous images, we first employ a convolutional gated recurrent unit to process spatiotemporal driving information on the basis of U-Net. Second, the pyramid split attention (PSA) module is used to enhance or suppress the obtained feature expressions. Finally, the skip connection is used to fuse the features of different scales encoded by each stage with the features processed by PSA and gradually restore to the original image size. Experiments on the TuSimple dataset demonstrate that our model outperforms representative lane line detection networks in challenging driving scenes, with an F₁-measure of up to 94.302%.

Under the complex environment, in the process of sensing and understanding the surrounding scene with an unmanned system, the traditional single-source panorama provides limited information and is prone to interference from lighting, it is difficult to capture hidden objects, and the user’s three-dimensional (3D) experience is poor. To solve this problem, we propose a real-time panoramic map modeling method based on multisource image fusion and 3D rendering with intensity, hue, saturation transform and wavelet technology, open graphics library. Five general image sharpness evaluation methods are used to analyze the single-source and multisource close-range and distant images during the day and night. The results show that in the daytime, the sharpness of the multisource image is significantly improved compared with the single-source visible light image, especially the energy gradient evaluation can reach 3.4 times. Similarly, compared with the single-source infrared image, the multisource image at night also improve significantly, the energy gradient evaluation can reach 1.4 times. This conclusion is verified by the subsequent experiments of several indoor scenes, the 3D rendering processing technology of panoramic map is realized, which improves user experience of 3D stereoscopic vision, and will provide multisource image fusion and 3D rendering technical support for unmanned real-time panoramic map modeling.

Human perception of visual stimulus and its physical characteristics have a nonlinear-logarithmic relation as stated in Fechner’s law, which is a psychophysical law of perception (i.e., subjective sensation). Inspired by Fechner’s law, we use the salient differences among pixels in images as the physical characteristics to mimic the human pattern perception and propose a Fechner local image descriptor (FLID) for face image representation. FLID is a nonlinear descriptor, so the represented image is more hierarchical than the general linear one. In addition, a threshold method is used to divide the features into four different intervals, which can effectively reduce the effects of noise and illumination. However, considering that FLID only extracts the local features of single-scale blocks, we incorporate multi-scale block into FLID and propose a multi-scale block Fechner local image descriptor (MB-FLID) to extract more global structural features. In the experimental part, we verified the performance of FLID and MB-FLID on the Face Recognition Technology (FERET), extended Yale-B, Olivetti Research Laboratories, Yale, and AR face databases, and compared them with a host of other local feature descriptors. The experimental results show that the proposed FLID and MB-FLID outperform the compared descriptors.

The position of blind lanes must be correctly determined in order for blind people to travel safely. Aiming at the low accuracy and slow speed of traditional blind lanes image segmentation algorithms, a semantic segmentation method based on SegNet and MobileNetV3 is proposed. The main idea is to replace the coding part of the original SegNet model with the feature extraction part of MobileNetV3 and remove the pooling layer. Blind lanes images were collected through online search and self-shooting, and then the data were manually marked by LabelMe software and trained on TensorFlow deep learning framework. The experimental results show that the improved model has high segmentation accuracy and recognition speed. The pixel accuracy of blind lanes segmentation is 98.21%, the mean intersection over union is 96.29%, and the average time for processing a 416 × 416 image is 0.057 s, which meets the real-time requirements of the blind guidance system.

The Editor-in-Chief and the publisher have retracted this article. An investigation uncovered evidence of compromised peer review and determined the paper to be out of the scope of the journal. The Editor and publisher no longer have confidence in the results and conclusions of the article.

SK, AJ, AN, GSY, and RK disagree with the retraction. PG and SR either did not respond or could not be reached.

Accurate calibration of camera internal parameters is an important step in realizing three-dimensional object reconstruction and target location. Large reprojection error is a frontier problem in the field of camera calibration. A real calibration image dataset and a reprojection error model are established. In combination with the fast iteration of elite opposition-based learning strategy and the population diversity of sparrow search algorithm, the calibration parameters were optimized to improve the accuracy of calibration parameters and reduce the reprojection error. Experimental results demonstrate the effectiveness of the proposed algorithm. It can effectively reduce the reprojection error.

Three-dimensional (3D) reconstruction of unmanned aerial vehicle (UAV) images has garnered a lot of attention with the rapid growth of UAV technology. The crucial technique for large-scale 3D reconstruction of images taken from UAV is to get accurate and drift-free state estimation of UAV. Vision-based simultaneous localization and mapping (SLAM) system is widely used in this field because of the function of robot localization and unknown environment mapping. However, the current visual SLAM systems are vulnerable to issues including feature extraction challenge, mismatching, cumulative drift, and tracking lost in complicated and low-texture environment. To cope with the aforementioned challenges, we build a stereo vision SLAM framework based on ORB-SLAM, which fuses global positioning system (GPS) information and dense matching results based on deep learning to achieve accurate and drift-free pose estimation and 3D reconstruction of UAV images. We utilize cutting-edge method to create pixel-wise dense matching of adjacent frames and stereo frames and make full use of them. Then, to minimize the accumulated drifts, global GPS data are combined with pose graph optimization, which aligns local estimation with the global coordinate. Finally, the globally accurate and drift-free pose estimations are used to build a sparse point cloud map. Patch-based multi-view stereo algorithm is implemented to construct a dense map based on UAV images for reasonable results. We evaluate the effectiveness and accuracy of our approach in both simulation and real-world experiments. Extensive experimental results demonstrate that our method achieves great results.

Object detection and tracking in maritime videos are crucial for marine management today. Nevertheless, due to the small size of ships in maritime video, current state-of-the-art algorithms are incapable of producing satisfactory results. We design an improved YOLOX (iYOLOX) and DeepSORT-based method to detect and track small objects in maritime videos to improve detection accuracy. Initially, the primary feature extraction capability of YOLOX’s backbone network is improved by adding the convolutional block attention mechanism. Second, we employ the context enhancement module to enlarge the receptive field while preserving the specific features. Finally, a parallel deformable convolution bifusion feature pyramid network is utilized for bidirectional feature extraction for small objects. iYOLOX is additionally combined with the DeepSORT algorithm to track targets. The experimental results show iYOLOX performed better when compared with the original YOLOX such that iYOLOX improves 3% average precision (AP) on the COCO2017 dataset, 4.7% AP_s for small-size objects, and 3.41% AP₅₀ on the LMD-TShip maritime video dataset. Furthermore, it indicates that the proposed algorithm effectively alleviates some object occlusion and camera shaking issues and improves the model’s utility and robustness.

Recent convolutional neural network (CNN)-based super-resolution (SR) studies incorporated non-local attention (NLA), to consider the long-range feature correlations and achieve considerable performance improvement. Here, we proposed an innovative NLA scheme called multi-scale NLA (MS-NLA) that computes NLAs at multiple scales and fuses them. To effectively fuse NLAs, we also proposed two learning-based methods and analyzed their performance on a recurrent SR network. Furthermore, the effect of weight sharing in the fusion methods is analyzed as well. In 2 × and 4 × SR experiments on benchmark datasets, our method had higher PSNR values of 0.295 and 0.148 dB on average than those using single-scale NLA and cross-scale NLA, respectively, and produced visually more pleasing SR results. The weight sharing had a limited but positive effect, depending on datasets. The source code is accessible at https://github.com/Dae12-Han/MSNLN.

Deep feature extraction and aggregation are critical components for lightweight single-image super-resolution networks. However, advanced feature extraction approaches like the reparameterization used in edge-enhanced feature distillation network increase the reparameterization structure to seven branches, which introduces additional training time and limits devices with strong parallel computing power, such as graphics processing unit. Simultaneously, commonly used feature aggregation methods, such as global and local feature aggregation, do not sufficiently exploit hierarchical features extracted from different deep feature extraction blocks. To address the above limitations, we propose a novel reparameterized branching block structure named spatially and channel diversified branching block. It employs three components with different receptive fields and two depthwise convolutions with different kernel sizes to increase spatial and channel diversity, enrich the reconstruction details, and significantly reduce the training time cost of the branches. Meanwhile, we analyze the memory consumption of generic aggregation methods and propose an adaptive hierarchical aggregation (AHA) method. It adopts a set of trainable parameters and channel attention mechanisms to implement a selective aggregation approach, further enhancing the feature aggregation’s generalization ability. Based on the proposed feature extraction and aggregation methods, we construct a spatial and channel diverse network (SCDNet). Extensive experiments across five benchmark datasets demonstrate the effectiveness of spatial and channel features in feature extraction, with AHA showing perfect aggregation ability. Our method provides superior performance over state-of-the-art single-image super-resolution methods, including reparameterized branch structures and other common approaches, in terms of both standard metrics and visual effects.

Scale variation is one of the challenges of object detection. Most state-of-the-art object detectors depend on feature pyramid networks (FPN) for multiscale learning to deal with this problem, in which feature fusion is an essential operation. However, feature fusion does not sufficiently address the difficulty of the detection task. This paper presents an enhancement-fusion feature pyramid network (EFPN) to obtain reliable object representations for object detectors. Specifically, it contains a feature enhancement module (FEM) and a bottom-up path module (BPM). The FEM is used to eliminate the negative impact of the uneven distribution of object scales on the model performance. Then, a BPM is proposed to address the fusion inconsistency in the FPN. Additionally, an attention module (A_c) is added to eliminate the information loss in the bottom-up aggregation process. EFPN is evaluated by combining it with state-of-the-art detection methods. Extensive experimental results on two datasets MS-COCO and VOC2007 demonstrate the effectiveness of the proposed method.

Automated plant disease identification is one of the significant tasks in the agricultural sector. With the advent of deep-learning models, classification and segmentation have been completely automated using a convolutional neural network (CNN). This paper proposes the UV-Net model, with a recurrent encoder and an up-sampling decoder for multiclass image segmentation and generation of masks. The improvement of the segmentation model is the stacking of conv2D layers before the down convolution operation, concatenation of features extracted from the convolution process, up-sampling along with transpose convolution in the expansion path, and exponential linear unit (ELU) activation. This model is combined with an optimized, pretrained, and self-trained CNN for generating the classes of diseased leaves. The classification is further enhanced by concatenating the high-level features extracted from the pretrained CNN model along with the low-level features from the encoder part of the UV-Net model. The features are concatenated using the hybrid wind-driven particle swarm optimized (WDPSO) deep feature selective concatenation mechanism (DFSCM). The proposed model is trained and the performance is tested on multiclass diseases of tomato plants like bacterial spot, early blight, leaf mold, and target spot. The proposed architecture is also compared with some of the well-known CNN models such as FCN, FPN, U-Net, V-Net, and Mask R-CNN. The model proves to be a robust system for detecting diseases from images of varying sizes, shapes, and lighting. The proposed model achieved an average training accuracy of 98.20% with a dice coefficient of 0.8938 and an Intersection over union coefficient (IoU) of 0.7458. The model also generates accurate masks from which the intensity and pattern of diseases could be identified for earlier prevention and treatment of diseases.

Limited by the network bandwidth, three-dimensional (3D) point cloud needs to be efficiently compressed before transmission. As one of the three attribute coding methods adopted in the geometry-based point cloud compression (G-PCC) standard developed by MPEG, predicting transform (PT) has received increasing attention. To further improve the coding efficiency of PT, we propose a rate-distortion optimized quantization (RDOQ) in which an additional option for quantization results is added by setting the quantized residuals to zero forcedly. Rate-distortion optimization is then used to determine whether the final quantized residual should be zero or nonzero. Experimental results show that average BD rates of −0.5 % , −3.0 % , and −3.0 % can be achieved for Luma, Chroma Cb, and Chroma Cr components, respectively, with negligible increment of time complexity.

Effective Bayesian network structure learning algorithms, such as the Peter and Clark (PC) algorithm, must be designed and used with data integrity as a top priority. Deep learning models can be jointly built by thousands of participants; thanks to the innovative distributed learning technology known as federated learning. This research proposes a technique for the detection of data poisoning attacks along with enhancing secure data transmission and routing by utilizing the integration of the PC algorithm and federated learning. Data integrity of the network is likewise enhanced using the Convergence of the PC algorithm. Detection of attacks in established secure data transmission is carried out using Bayesian adversarial federated learning. We provided an optimization-based model poisoning approach and introduced adversarial neurons into the redundant area of a neural network by assessing model capacity. It should be highlighted that while those redundant neurons have little relevance to the primary goal of federated learning, they are crucial for poisoning attacks. Numerical tests show that the suggested approach can get beyond defense mechanisms and have a high attack success rate.

Computational color constancy (CCC) consists of computing color estimates of the ambient scene illumination and using them to remove unwanted chromatic distortions. Much research has focused on illuminant estimation for CCC on single images, with few attempts of leveraging the temporal information intrinsic in sequences of correlated images (e.g., the frames in a video), a task known as temporal color constancy (TCC). The state-of-the-art for TCC is temporal color constancy network (TCCNet), a deep-learning architecture that uses a convolutional long short-term memory network for aggregating the encodings produced by convolutional neural network submodules for each image in a sequence. We extend this architecture with different models obtained by (i) substituting the TCCNet submodules with cascading convolutional color constancy (C4), the state-of-the-art method for CCC targeting images and (ii) adding a cascading strategy to perform an iterative improvement of the estimate of the illuminant. We tested our models on the recently released TCC benchmark and achieved results that surpass the state-of-the-art. Analyzing the impact of the number of frames involved in illuminant estimation on performance, we show that it is possible to reduce inference time by training the models on a few selected frames from the sequences while retaining comparable accuracy.

Real-time emotion detection based on facial expression is an innovative research field that has been applied in several areas, such as health, human–machine vision, and autonomous safety. Researchers in object detection are involved in developing methods to interpret, code facial expressions, and extract these features to be better predicted by machines. Furthermore, the success of deep learning with different architectures is exploited to achieve better performance. But these methods drastically fail in excessive sweating in different health conditions. We aim to create a dataset in different health conditions and detect facial emotion using the encoder and decoder-based deep learning methodology. The proposed architecture and the dataset present the progress made by comparing the other proposed methods and the quantitative and qualitative results obtained. The major benefit of our study is to enhance the emotion detection efficiency with other proposed methods and real-time applications for different health conditions. We propose the application of feature extraction of facial expressions with an end-to-end attention module-based fusion network for detecting different facial emotions (happy, angry, neutral, surprised, etc.) with an accuracy of 99.68%. The proposed system depends upon the human face; as we know, the face reflects human brain activities or emotions.

Most of the existing Siamese network-based trackers achieve the similarity matching problem through the cross-correlation of convolutional features between the template branch and the search branch. However, the method of using the overall feature of the target as the convolution kernel to cross-correlate with the search area contains a lot of background information, which will adversely affect the tracking. To solve this problem, we propose a feature fusion and saliency-attention Siamese network (MSSiamCAR) for object tracking, adopting a part-based tracking strategy to track discriminative local salient regions in objects. This algorithm uses the ResNet-50 network to extract the target features. To solve the loss of the shallow part of the target due to the deepening of the network, a multilayer feature is proposed. The fusion module enhances the feature extraction ability of the network for the target. Second, a saliency capture module is proposed to obtain the local saliency of the target. These saliency are robust to interference factors. The saliency interaction module is designed, and the graph attention mechanism is used to establish an effective connection between the saliency, and the target feature information is propagated to the search feature so that the tracker has more discriminative ability to the target feature. Finally, the region proposal network is used to perform operations, such as classification, regression, and centrality calculation. Experiments on challenging benchmarks, including OTB-100, VOT2018, large-scale single object tracking (LaSOT), generic object tracking-10k (GOT-10k), and unmanned aerial vehicles 123 (UAV123), demonstrate that our proposed tracker performs favorably against state-of-the-art trackers.

This erratum corrects an error in Eq. 35.