Unsupervised visible-infrared person re-identification (USVI-ReID) plays a crucial role in computer vision. The key challenge of USVI-ReID is to learn the discriminative features of images and establish cross-modal correspondence without using class labels. We propose a two-stage contrastive learning method for USVI-ReID. The first stage is instance-wise contrastive learning for learning a discriminative model. The learned discriminative model is transferred to the second stage for clustering operation, thus forming category-level supervision and promoting the execution of cluster-wise contrastive learning. Besides, a progressive training strategy is proposed to gradually shift the model’s attention from instances to clusters. Extensive experiments on two public datasets SYSU-MM01 and RegDB demonstrate the effectiveness of the proposed method.

Convolution neural networks (CNNs) and transformers are good at extracting local and global features, respectively, whereas both local and global features are important for the no-reference image quality assessment (NR-IQA) task. Therefore, we innovatively propose a CNN–transformer dual-stream parallel fusion network for NR-IQA that can simultaneously extract local and global hierarchical features related to image quality. In addition, considering the importance of saliency in NR-IQA, a saliency-guided CNN and transformer feature fusion module is proposed to fuse and optimize the hierarchical features extracted by the dual-stream network. Finally, the high-level features of the dual-stream network are fused through the local and global cross-attention module to better model the interaction relationship between local and global information in the image, and the quality prediction module containing evaluation and weight branches is used to obtain the quality score of distorted images. To comprehensively evaluate the performance of our model, we conducted experiments on six standard image quality assessment datasets, and the experimental results showed that our model has better quality prediction performance and generalization ability than previous representative NR-IQA models.

The hippocampus, a crucial structure in the brain, plays a significant role in the early diagnosis of brain disorders such as Alzheimer’s disease through its structural and volumetric changes. To address the medical challenge of accurately segmenting the hippocampus, we propose a lightweight hybrid segmentation network called a parallel cascaded feature reconstruction network (PCFR-Net). This network integrates the advantages of global self-attention and local convolution while utilizing fewer model parameters. Specifically, we introduce a feature reconstruction (FR) module and a multibranch asymmetric residual attention module aimed at accurate segmentation of hippocampus magnetic resonance imaging. The model combines the strengths of the transformer in capturing long-distance relationships and adapting to irregular shapes, as well as the FR block, which can reduce the redundancy in space and channels during feature extraction, and then reconstructs feature maps to enhance the representative feature learning. In addition, the multibranch residual attention module employs the asymmetric residual convolution block, enabling fine-grained feature extraction along the length, width, and depth directions at multiple scales. Remarkably, the proposed PCFR-Net achieves a Dice similarity coefficient (DSC) of 92.74% and an Intersection over Union (IoU) of 86.5% on the Medical Segmentation Decathlon, as well as a DSC of 93.86% and an IoU of 89.29% on the Alzheimer’s Disease Neuroimaging Initiative dataset.

Real-time object detection has consistently confronted the challenges of the trade-off between inference latency and detection accuracy, especially in dense scenarios. The attention mechanism is widely used in improving detection accuracy by obtaining the awareness of networks on the critical region of the raw inputs. Previous works proposed to generate attention along different feature dimensions to strengthen selective attention, but the optimal leverage of attention is still a challenging problem. We argue that the extra trainable parameters and lacking multi-scale capabilities induced by current attention designs make it difficult to detect dense targets accurately in real time. We propose a novel parameter-free attention called guided supervised attention (GSA). Instead of using trainable parameters, GSA innovatively explores the objectness from the detector’s output to generate a reliable spatial attention mask. With the inherent characteristic of label assignment, GSA is also scale-aware, thus improving the multi-scale capability for dense target scenarios. Combined with deep supervision techniques, such a parameter-free design ensures a low training and inference cost, thus realizing a good trade-off between speed and accuracy. Our experiments empirically prove that the one-stage method applied with GSA on the head obtains an average precision (AP) improvement of 0.2% to 1.1% on the CrowdHuman benchmark with negligible additional computational cost. Furthermore, with GSA, the YOLO series achieves an AP of 92.88% CrowdHuman, which is found to have the best AP among all the proposal-based methods, presenting a competitive performance while maintaining real-time inference. The study of the GSA introduces a novel application of objectness information in object detection.

In the realm of video understanding and analysis, solely relying on the appearance features of individuals in video frames significantly falls short of enhancing the accuracy of group activity recognition. The comprehensive utilization of various feature information present is deemed crucial, playing a pivotal role in understanding group activities. Consequently, a three-stream architecture model for feature learning is proposed. This model not only considers the human appearance features and available scene-level context information for group activity recognition within videos but also emphasizes the model’s perception of individual motion, uncovering valuable information about motion features. Integrating appearance, motion, and scene-level context information affords a more comprehensive and rich representation of individual features. Ultimately, these combined features are employed in relation analysis to better predict group activities. The effectiveness of the proposed method is validated on two benchmark datasets, volleyball and collective activities, demonstrating its efficacy for the task.

Non-local attention (NLA) is a powerful technique for capturing long-range feature correlations in deep single-image super-resolution (SR). However, NLA suffers from high computational complexity and memory consumption, as it requires aggregating all non-local feature information for each query response and recalculating the similarity weight distribution for different abstraction levels of features. To address these challenges, we propose a novel learnable collaborative attention (LCoA) that introduces inductive bias into non-local modeling. Our LCoA consists of two components: learnable sparse pattern (LSP) and collaborative attention (CoA). LSP uses the k-means clustering algorithm to dynamically adjust the sparse attention pattern of deep features, which reduces the number of non-local modeling rounds compared with existing sparse solutions. CoA leverages the sparse attention pattern and weights learned by LSP and co-optimizes the similarity matrix across different abstraction levels, which avoids redundant similarity matrix calculations. The experimental results show that our LCoA can reduce the non-local modeling time by about 83% in the inference stage. In addition, we integrate our LCoA into a deep learnable collaborative attention network (LCoAN), which achieves competitive performance in terms of inference time, memory consumption, and reconstruction quality compared with other state-of-the-art SR methods. Our code and pre-trained LCoAN models were uploaded to GitHub ( https://github.com/YigangZhao/LCoAN) for validation.

Foreign object debris (FOD) on airport runways poses a significant threat to airline safety, making the study of FOD detection algorithms crucial for aviation security. Addressing the challenges of FOD detection, such as the small pixel occupancy of FODs in images, their lack of distinctive features, and their susceptibility to misdetection or omission, we propose an enhanced FOD detection method based on YOLOv5s, named VA-YOLO. The VA-YOLO method incorporates several key modifications to improve detection accuracy. First, a small target detection layer is added in the multi-scale fusion part to enhance the identification of small-sized targets, whereas the large target detection layer is removed to reduce computational load. Second, a general but efficient MobileViT block is introduced in the backbone section, replacing convolution with transformers to learn global representations. Third, the SKAttention module is integrated into the neck section to focus more on the target’s feature information. In addition, a hybrid loss function named normalized complete-IoU (NCIoU), is proposed, which combines the advantages of complete-IoU and normalized Gaussian Wasserstein distance to enhance detection accuracy. Comparative and ablation experiments were conducted on our self-constructed FOD-Z dataset. The experimental results, under the assumption of maintaining real-time performance, demonstrate that VA-YOLO achieves 99.2% mAP@0.5, 93.6% mAP@0.75, and 84.3% mAP@0.5:0.95. Compared with the original YOLOv5s method, the detection accuracy is improved by 1.4%, 3.5%, and 3.4%, respectively. Thus, VA-YOLO effectively addresses the issues of misdetection and omission of FODs on airport runways.

Current semantic segmentation algorithms are often burdened by high computational complexity and inadequate boundary localization accuracy in complex scenarios of industrial manufacturing. To achieve lightweight semantic segmentation with high accuracy, we propose a semantic segmentation algorithm named L-DeepLabV3+. First, a CB-MobileNetV3 is proposed as the backbone network, which reduces the computational burden. Second, we propose a DSCA_ASPP module to compensate for the loss of some detailed information in the encoder and enhance the perception of boundary information. Within the DSCA_ASPP module, we propose dilated separable convolution to address the issue of dilated convolutional meshing. Finally, we introduce the Haar Wavelet Transform with Deepwise Separable Convolution (HWTS) module as a replacement for the 3×3 convolution in the decoder. It consists of a deepwise separable convolution and Haar wavelet transform module, which further reduces the number of model parameters and improves the feature representation in the decoder. Experimental results demonstrate that L-DeepLabV3+ achieves an 82.7% mIoU on the PASCAL VOC2012 dataset, utilizing only 2.73 million parameters and a floating-point computation amounting to 43.015 G. The L-DeepLabV3+ model has ∼4% of the number of parameters compared to the traditional DeepLabV3+, resulting in a significant reduction in computational burden. Furthermore, L-DeepLabV3+ outperforms common semantic segmentation algorithms while using fewer model parameters and enhancing segmentation performance. Moreover, L-DeepLabV3+ exhibits excellent performance in detecting defects in industrial ceramic substrates.

In scenarios with strong light sources, images captured by surveillance cameras often exhibit lens flare, which can significantly blur or even completely obscure the details of the surveillance footage, thereby reducing the accuracy of the monitoring data. Existing algorithms for removing lens flare often result in blurred edge details in the damaged areas of the image and suffer from low computational efficiency. To address these issues, this paper proposes an algorithm for removing lens flare from images by integrating semantic information. Based on the principles of optical flow, this algorithm constructs a feature map fusion model to learn and enhance the semantic information between adjacent feature maps, thereby capturing the semantic information in the lens flare areas and restoring the lost edge details of the damaged regions. In addition, an improved channel attention mechanism is introduced, which uses a scaling factor to normalize the features of irrelevant background areas and lens flare, reducing the weight of features from irrelevant areas, and specifically extracting semantic features from the lens flare area to improve the computational efficiency of the algorithm. The feature map fusion model and the improved channel attention mechanism are embedded into the skip connections and decoder pathways of U-Net, focusing on fusing high and low-level semantic information and specifically targeting the lens flare areas, thus effectively removing lens flare from images. Experiments on the Flare7K++ dataset from Nanyang Technological University’s S-Lab show that our proposed algorithm achieves a peak signal-to-noise ratio of 27.7643 dB, a structural similarity index measure of 0.9739, and a reduced learned perceptual image patch similarity of 0.0437, outperforming existing methods and demonstrating the effectiveness of our approach in the field of lens flare removal.

The fuzzy K-nearest neighbors (FKNN), a renowned fuzzy classifier, has attained commendable performance. We introduce adaptive fuzzy contribution classification (AFCC), a novel fuzzy classifier that diverges from FKNN’s reliance on majority voting. Instead, AFCC employs an improved category contribution rate (ICCR) to delineate the hierarchical relationship between samples and categories. ICCRs quantify the contribution of a sample in reconstructing all categories, exhibiting resilience to noise. In the presence of noisy samples, AFCC adjusts all ICCRs concurrently, preserving their differentials and thus the hierarchical relationship between samples and categories. In AFCC, a precise subordination degree for each sample across all categories is determined, bridging the gap between samples and their respective categories. With this subordination degree established, the decision subordination degree for each test sample toward all categories can be derived using relevant samples. The category with the highest decision subordination degree is then selected. Notably, AFCC operates as a parameter-free classifier. Experiments have been extensively conducted on various datasets, including the Yale, Olivetti Research Laboratory (ORL), and Aleix Martinez and Robert Benavente (AR) face databases, as well as the Center for Pattern Recognition and Machine Intelligence (CENPARMI) handwritten numeral database. The outcomes unequivocally demonstrate that the recognition performance of AFCC surpasses that of other methods. Notably, in the AR face database experiments, AFCC achieves remarkable recognition accuracies, reaching 99% for images exhibiting expression variations and an impressive 100% accuracy for images subjected to illumination changes, showcasing its potential for applications in areas requiring robust and efficient pattern recognition, such as facial recognition, handwriting analysis, and other image-based classification tasks.

Because the reconstructed images of existing remote sensing image super-resolution algorithms have problems such as lack of texture information and loss of high-frequency details, we propose a remote sensing image super-resolution reconstruction algorithm based on a multiscale residual network. First, in the shallow feature extraction stage, we design a tandem convolution group containing different scale convolution kernels to improve the model’s ability to extract shallow information. Second, we design an enhanced attention residual block to extract deep features so that the model can capture more texture information. To further improve the performance of the model, during the feature enhancement stage, we design a pyramid-polarized attention convolution module, which uses a more efficient attention mechanism and pyramid convolution to capture long-distance dependencies in the image. To enhance the stability of the model, we use a Charbonnier loss function based on pixel loss. Finally, we use the Adan optimizer to optimize the model to improve the convergence of the model and achieve better reconstruction results. The experimental results show that our algorithm can better reconstruct the texture detail of remote sensing images and, compared with existing super-resolution reconstruction algorithms, the proposed method can achieve better subjective visual effects in multiple remote sensing image datasets.

Detecting small objects in unmanned aerial vehicle (UAV) pictures is a vital but demanding task in the field of computer vision due to factors such as object size, image degradation, and computational resource limitations. Most existing algorithms improve object detection accuracy by improving feature fusion structures and incorporating single-layer attention modules. However, the weight of single-layer attention is computed only based on the original input, which leads to insufficient exploration of correlations among cross-layer features. To remedy the above issues, we design a novel multi-scale UAV image object detection model called CDANet. First, to fully utilize the scattered detail features in the shallow layers, a cross-layer attention module named the enhanced guided attention module is designed. Second, a simplified depth-wise separable dilated spatial pyramid pooling-fast module is built to overcome the shortcomings of max-pooling operation during feature fusion. Third, an efficient coordinate guidance module is designed as the basic module for guiding the inter-channel information interaction. Fourth, a cross-spatial learning encoder module is designed to encode cross-channel information with an efficient multi-scale attention module. Last, to optimize the learning process by dynamically regulating the attention toward diverse objects, an advanced loss function Wise-IoU v3 is introduced. The results show that, on the VisDrone2021-DET dataset, the average precision and mean average precision of the proposed method are superior to YOLOv8, which are respectively improved by 3.9% and 4.9%.

Remote sensing image plays an important role in human activities and environmental monitoring, and it is often necessary to perform super-resolution (SR) reconstruction for clear effects. However, the absence of multi-scale features ignores the correlation among different feature layers, which can affect reconstruction. We propose a micro-pyramid SR generative adversarial network (MSRGAN) method for the SR reconstruction. First, the micro-pyramid block was introduced to extract multi-scale feature information. Then, a layer-by-layer additive method was proposed to fuse the feature information of different scales to improve the network performance. In addition, a parallel attention unit feature selection module is designed to enhance the correlation relationship among image features and improve the utilization of high-frequency features. Finally, the experimental results show that the proposed MSRGAN method has better reconstruction ability than the Bicubic, super-resolution convolutional neural network, efficient sub-pixel convolutional neural network, super-resolution using very deep convolutional network, residual channel attention network, and super-resolution generative adversarial network methods.

Low-light object detection is critical in computer vision, with widespread applications in nighttime surveillance and autonomous driving. Low-light environments pose several challenges, including increased image noise, insufficient contrast, blurriness, and color distortion, all of which affect target discernibility, particularly for small objects. Furthermore, the requirement for real-time processing adds to the complexity of image handling in these conditions. We introduce a novel approach to improving low-light target detection accuracy while maintaining a compact model. We enhance the YOLOv7-tiny network through several key innovations. We replace the simplified efficient layer aggregation network (ELAN-L) with the enhanced ELAN with fused diverse branch block (ELAN-DBB), which better integrates features from various branches and layers to improve information extraction from low-light images. In addition, we incorporate a new neck structure featuring the efficient cross-stage partial network module integrating Convolutional Block Attention Module and Deformable Convolutional Networks (VoVGSCSP-DCN-CBAM) module. This module combines attention mechanisms and depth-separable convolutions to enhance feature fusion and improve detection performance in low-light conditions. To address spatial awareness, we use CoordConv in place of 1×1 convolutions within the feature pyramid module, which enhances the model’s ability to capture spatial location information crucial for accurate detection. Furthermore, we introduce a specialized pruning module, DBB-pru, which, alongside comparing three pruning methods, results in a 35% reduction in model size and a 3.4% increase in accuracy compared with YOLOv7-tiny. Our approach markedly enhances the precision of low-light target detection while offering a more streamlined and efficacious model for practical deployment in detection applications.

Text-to-image (T2I) generation techniques have shown promising results using deep learning models. The recent T2I methods mainly use complex and stacked structures to maintain text-image consistency or focus on the generation of global text information while overlooking the finer details. In addition, brocade images differ from real-world images in various aspects, such as their fine texture and complex pattern elements (totem elements). To solve the problems, we first build a text-image dataset that specifically focuses on Chinese brocade. Moreover, by developing an extra pattern discriminator and the multi-scale feature fusion module, we proposed the brocade dual-discriminator generative adversarial network (BDD-GAN). BDD-GAN addresses the challenges associated with text-image consistency, as well as capturing the intricate textures and unique patterns in Chinese brocade-generated images. To fully verify the performance and effectiveness of the proposed method, we provided ablation studies and comparisons with previous works. Experimental results indicate that our method is capable of synthesizing Chinese brocade images that are higher in quality and totem details than those produced by previous algorithms.

As the application of urban road extraction becomes more widespread, the challenges of segmentation errors and embedding large models into hardware become more complex. To solve these problems, an algorithm called attention mechanism and lightweight network fusion high-resolution network (AMLN-HRNet) is proposed. The network includes a lightweight convolution called deep sparse channel and spatial encoding convolution (DSCEConv) for road extraction, which greatly reduces the number of parameters in the model. To make the extraction of roads more accurate, an attention mechanism called lightweight dynamic weighted is designed. In addition, a parameterless attention mechanism is introduced to make the model properly combine the spatial correlation and topological structure of the road to improve extraction accuracy. Through experimental results, AMLN-HRNet can effectively balance the speed and accuracy of the model.

We propose a new approach to multi-view subspace learning termed multi-view latent space learning via adaptive graph embedding (MvSLGE), which learns a latent representation from multi-view features. In contrast to most existing multi-view latent space learning methods that only encode the complementary information into the latent representation, MvSLGE adaptively learns an adjacent graph that effectively characterizes the similarity among samples to provide additional regularization to the latent representation. To extract neighborhood information from multi-view features, we introduce a novel strategy that constructs one graph for each view. Subsequently, the learned graph is approximately designed as a centroid of these graphs from different views, each assigned with different weights. Therefore, the constructed latent representation not only incorporates complementary information from features across multiple views but also encodes the similarity information of samples. The proposed MvSLGE can be solved by the augmented Lagrangian multiplier with the alternating direction minimization algorithm. Plenty of experiments substantiate the superiority of MvSLGE on a variety of datasets.

Multi-image hiding, which embeds multiple secret images into a cover image and is able to recover these images with high quality, has gradually become a research hotspot in the field of image steganography. However, due to the need to embed a large amount of data in a limited cover image space, issues such as contour shadowing or color distortion often arise, posing significant challenges for multi-image hiding. We propose StegaINR4MIH, a implicit neural representation steganography framework that enables the hiding of multiple images within a single implicit representation function. In contrast to traditional methods that use multiple encoders to achieve multi-image embedding, our approach leverages the redundancy of implicit representation function parameters and employs magnitude-based weight selection and secret weight substitution on pre-trained cover image functions to effectively hide and independently extract multiple secret images. We conduct experiments on images with a resolution from three different datasets: CelebA-HQ, COCO, and DIV2K. When hiding two secret images, the PSNR values of both the secret images and the stego images exceed 42. When hiding five secret images, the PSNR values of both the secret images and the stego images exceed 39. Extensive experiments demonstrate the superior performance of the proposed method in terms of visual quality and undetectability.

Underwater images currently suffer from color tone bias, reduced contrast, and blurring, which significantly hinder their use in advanced visual tasks. To address these visual quality issues, we present an underwater image restoration method guided by a physics model, integrating generative adversarial network and Unified Transformer frameworks. The network is divided into three parts: a physical model parameter estimation network (Par-subnet), an encoder–decoder, and discriminators. Initially, the parameter estimation network is used to compute the parameters required by the physical model, resulting in a preliminarily restored underwater image through inversion. We enhance the encoder–decoder network by incorporating the Unified Transformer and Reinforced Swin-Convs Transformer, thereby improving its capacity to capture both global and local features effectively. Simultaneously, a degradation quantization module is used to assess the degraded regions of the underwater image, further improving the restoration performance of the encoder–decoder. Given the limited availability of real underwater reference images, we employ the reliable bank mechanism to filter the best-quality results from the model output images and the reference images for training the discriminators. This process facilitates the transition from supervised to semi-supervised learning. The comparative experimental findings indicate that our methodology outperforms existing methods across both quantitative and qualitative metrics on multiple benchmark datasets, fully verifying the reliability and efficacy of our method.

Semantic segmentation is a fundamental and important task in the field of remote-sensing images. However, most convolutional neural network (CNN)-based methods often produce undesirable results of blurry object boundaries as convolution operations may inherently fail to retain the edge information while expanding the receptive field. Besides, CNNs struggle to capture subtle relationships between distant parts of an image. This limitation can be particularly detrimental to semantic segmentation as it requires a comprehensive understanding of the global image context. Transformers, nevertheless, excel at processing long-range dependencies. To address the above issues, we propose a novel three-branch network with three new modules, called GDBNet. The Swin-Transformer-based global (G) branch is for capturing global context information. The ResNet-based detail (D) branch and boundary (B) branch for processing local features and edge information. The three modules are feature preservation module (FPM), spatial interaction module (SIM), and balance feature module (BFM). FPM can maximumly retain the context information during the downsampling stage of Swin Transformer in the global branch, whereas SIM and BFM can fuse the information from different branches in a boundary-oriented manner. Experiments show that GDBNet outperforms state-of-the-art methods quantitatively and qualitatively on Vaihingen, Potsdam, and WHDLD datasets, reaching 71.87%, 75.64%, and 62.70% accuracy in MIOU metric, respectively, and generating visually better clearance and coherence of boundaries.

The accurate reconstruction of topology and texture details of three-dimensional (3D) objects from a single two-dimensional image presents a significant challenge in the field of computer vision. Existing methods have achieved varying degrees of success by utilizing different geometric representations, but they all suffer from limitations when accurately reconstructing surfaces with complex topology and texture. Therefore, this study proposes an approach that combines the convolutional block attention module (CBAM), texture detail fusion, and multimodal fusion to address this challenge effectively. To enhance the model’s focus on important areas within images, we integrate the CBAM mechanism with ResNet for feature extraction. Texture detail fusion plays a crucial role as it effectively captures changes in the object’s surface while multimodal fusion improves the accuracy of predicting the signed distance function. We have developed an implicit single-view 3D reconstruction network capable of retrieving topology and surface details of 3D models from a single input image. The integration of global, local, and surface texture features is a significant advancement that improves shape representation and accurately captures surface textures, filling a crucial gap in the field. During the process of reconstruction, we extract features that represent global information, local information, and texture variation information from the input image. By utilizing global information to approximate the shape of the object, refining shape and surface texture details through the utilization of local information, and applying distinct loss terms to constrain various aspects of reconstruction, our method achieves accurate single-image 3D model reconstruction with detailed surface textures. Through qualitative and quantitative analysis, we demonstrate the superiority of our model over state-of-the-art techniques on the ShapeNet dataset. The significance of our work lies in its ability to enhance the quality of single-view implicit 3D reconstruction by effectively integrating these features, leading to a more robust and detailed reconstruction of 3D models from single images. The source code of this work is available online at https://github.com/YangPeppa/ITIR-Net.

Recently, vision transformers started to show impressive results that significantly outperform large convolution-based models. We propose a gating network for salient object detection. The Pyramid Vision Transformer serves as the backbone network of this gating network, learning global and local representations with its self-attention mechanism. Multi-level gating units are used to recover more details of the saliency map by establishing cooperation among different levels of features, thus improving the discriminability of the whole network. With the help of multi-level gating units, the valuable context information from the encoder can be optimally transmitted to the decoder. The pyramid pooling module collects high-level semantic information. Moreover, the semantic information of each level is integrated and decoded by the feature aggregation decoder. The experimental results on five challenging benchmark databases demonstrate that the proposed method achieves more favorable performance than the current state-of-the-art methods in terms of four evaluation criteria.

Recent advancements in 3D object detection using light detection and ranging (LiDAR)-camera fusion have enhanced autonomous driving perception. However, aligning LiDAR and image data during multimodal fusion remains a significant challenge. We propose a novel multi-modal feature alignment and fusion architecture to effectively align and fuse voxel and image data. The proposed architecture comprises four key modules. Z-axis attention aggregates voxel features along the vertical axis using self-attention. Voxel-domain deformable encoder improves context understanding with deformable attention to encode voxel features. Dual-domain deformable feature alignment uses deformable attention to adaptively align voxel and image features, addressing resolution mismatches. Finally, gated fusion utilizes a gating mechanism to dynamically fuse aligned features. The multi-layer design further enhances feature detail retention and improves dual-domain fusion performance. Experimental results show our method increases average precision by 2.41% at the “hard” difficulty level for cars on the KITTI test set. On the KITTI validation set, mean average precision improves by 1.06% for cars, 6.88% for pedestrians, and 1.83% for cyclists.

Underwater image enhancement has attained significant attention due to its applications in different fields such as marine engineering and aquatic robotics. An underwater image suffers from various deterioration issues such as color loss, degraded contrast, and poor visibility due to the attenuation and scattering of light and color. In recent years, many underwater image techniques have been proposed. However, these techniques introduce some undesired color tones with heavily attenuated color channels in deep water images and also do not work well for different types of hazy underwater images. To tackle these issues, we propose a color correction network and an underwater extended dark channel method that handles color cast issues and adaptively controls the different levels of hazy underwater images. The proposed model offers not only high-quality enhanced underwater images but also preserves the surface details. The experimental results are evaluated both qualitatively and quantitatively on three underwater benchmark datasets, namely, underwater image enhancement benchmark, underwater color cast removal and color correction, and enhancing underwater visual perception, and it is found that the proposed method outperforms the state-of-the-art approaches. In addition, the proposed method is generalized to remove the different underwater image degradation issues such as haze, low light, and color degradation. The code for the paper is available at https://github.com/sangeeta-rani/underwater-image-enhacement-using-extented-transmission-map.

Currently, with the development of deep learning techniques and large models, designing efficient network models has become one of the hot topics in research. In the field of image super-resolution reconstruction, although deep convolutional neural networks have made significant progress, the increase in network complexity has led to an increase in computational overhead and excessive consumption of computational resources on high-performance devices (e.g., GPU). To address this issue, a network for image super-resolution reconstruction based on partial convolution (Pconv) and an improved agent attention mechanism is proposed. By reducing redundant computations and memory access, the network can more effectively extract spatial features, significantly reducing computational complexity while maintaining superior performance. Through experiments comparing recent methods on public datasets in terms of performance metrics, the proposed network model demonstrates leading results in objective quantitative measures, promising to provide a more efficient and viable solution for image super-resolution reconstruction tasks.

Unsupervised domain adaptation for object detection leverages a labeled domain to learn an object detector generalizing to a different domain free of annotations. We propose efficient multi-scale attention, confidence mixing, augmentation, and combination (ECAC), an adaptive object detector learning method based on a region-level confidence sample mixing strategy. Compared with the current methods, our approach crops high-confidence detection regions from both the source and target domains, augments them, and combines them to generate composite samples. In addition, consistency loss is utilized to solve the domain adaptation problem. Furthermore, we introduce the efficient multi-scale attention (EMA) into the detector. To retain channel information and reduce computational overhead, EMA attention restructures part of the channels into the batch dimension and groups the channel dimension into multiple sub-features, ensuring spatial semantic features are evenly distributed within each feature group. EMA employs a shared 1×1 convolution branch from the CA attention module, along with a parallel 3×3 convolution kernel to aggregate multi-scale spatial structure information. This approach effectively enhances the model’s focus on region-level features by integrating local and global information with multi-scale parallel sub-networks and cross-spatial learning. For pseudo-label filtering, we progressively transition from a loose to a stricter confidence threshold. Initially, this allows more pseudo-labels, facilitating the detector’s learning of target domain representations. As training progresses, stricter thresholds are applied to select more reliable pseudo-labels, gradually filtering out inaccurate pseudo-detections. Our extensive experiments on three datasets demonstrate that ECAC achieves state-of-the-art performance on two of them. On the third dataset, our method improves the mean average precision by over 2% compared with the latest methods.

Deep learning methods have demonstrated excellent performance in semantic segmentation tasks across various scenarios. However, their performance remains inadequate for real-time applications, such as road scenes, prompting extensive research into real-time semantic segmentation. We proposed a network with a two-branch architecture that is a prevalent framework in this field. We first create an efficient backbone based on the U-like residual block we proposed to enhance multi-scale feature extraction capabilities. In addition, many two-branch networks have designed complex structures for fusing the feature maps of the two branches at the end of the networks. To simplify this process, our network designs a lightweight fusion mechanism that utilizes attention calculations to guide the fusing of features. We name this module as dual-guided attention. This fusion module operates in parallel, with each branch employing a fully connected layer with few neurons to determine the correlation of the flattened feature map, thereby executing the attention and feature fusion simultaneously. Extensive experiments on the Cityscapes and Cambridge-driving Labeled Video datasets show the effectiveness of our method.

Cracks, as a typical type of road defect, are important to be detected and repaired in time for transportation maintenance. However, fine-grained crack detection remains a challenge because of the slender shape, backdrop interference, and noise. We propose an automatic fine-grained crack detection algorithm based on a local enhancement self-attention mechanism and a deep semantic-guided multi-scale fusion strategy. The proposed network is built around an encoder–decoder framework. Inspired by the characteristic of global integrity and local continuity, we employ a Local Enhancement Self-Attention Block (LE Self-AB) in the encoder part. It captures the overall trend of cracks from a global perspective while focusing on the local details in the crack regions. The decoder network adopts a deep semantic-guided multi-scale fusion strategy. This strategy enables the model to benefit from both global and local information through cross-scale feature transfer and fusion. To validate the effectiveness and reliability of our approach, training and evaluation are conducted on the CrackTree260 dataset. The generalization of the model is tested on the CRKWH100 and CrackLS315 datasets. The experimental results demonstrate that the proposed method outperforms other advanced crack detection methods with an Optimal Dataset Scale of 0.919, 0.918, and 0.894 on three datasets, respectively.

Homomorphic encryption (HE) has emerged as a prominent research focus in the development of privacy-preserving biometric authentication systems, offering advantages over conventional methods such as cancelable biometrics and biometric cryptosystems. However, many existing systems relying on HE assume the integrity of the server, which may not always be guaranteed. In scenarios where the server behaves maliciously, there is a risk of false acceptances or rejections due to arbitrary computation results aimed at conserving computational resources. To mitigate this concern, we introduce a novel approach called privacy-preserving correlation-based multi-instance iris verification on malicious servers using Cheon-Kim-Kim-Song (CKKS) fully HE (PCMIA). PCMIA leverages CKKS HE to preserve the privacy of iris templates and one-bit checksum to check the correctness of the computed result. Moreover, we introduce discriminant correlation analysis (DCA), a feature-level fusion approach that improves the pair-wise correlations and reduces the between-class correlations. Extensive experimentation with benchmark iris databases demonstrates that PCMIA fulfills the requirements of the template protection scheme, as well as instilling trust in the comparator results.

Road damage detection plays a vital role in road safety maintenance. Minor road damage cannot often be identified efficiently because of the challenge of extracting damage information, which frequently lacks minor information. In addition, due to the irregular shape of alligator road damage, the performance of existing damage detection algorithms is seriously reduced. To solve the above problems, we propose an improved YOLOv5 model called YOLOv5-MCD for road damage detection. First, a multiscale dilated convolution module combined with an attention mechanism is proposed and used to replace the spatial pyramid pooling fast module in the neck of the original YOLOv5 model. Multiscale dilated convolution with a larger receptive field can be applied to obtain more damage feature information at different scales, and the attention mechanism aggregates the spatial and channel feature information of road damage images, allowing the model to pay more attention to damaged areas. Second, the cross-layer feature fusion method is proposed to merge more road damage features to enhance the original damage feature information. Finally, deformable convolution is used to improve the adaptability and performance of our method for complex alligator damage. Our method was tested and validated by the China-MotorBike RDD2022, and the F1-score and mAP@0.5 of our model were 93.5% and 95.5%, respectively. The experimental results show that our method can efficiently identify minor and alligator road damage and achieve better detection performance.

Infrared-visible image fusion (IVF) proves to be advantageous in conveying scene information comprehensively. Although convolution neural network (CNN)–based methods skillfully extract local information, their global information modeling capabilities remain insufficient. Conversely, attention-based methods excel in capturing global information but often neglect detailed features. Existing CNN-transformer methods have achieved excellent performance, but they still exhibit shortcomings in capturing image details. In response to these limitations, we introduce a Swin Transformer integrated with an invertible neural network (INN) and correlation coefficient (CC) assistance for the IVF network, denoted as SICFuse. SICFuse incorporates an attention enhancement module to enhance the extracted image information. For global information modeling, we employ Swin blocks, capitalizing on their characteristics to deliver multi-scale information and implementing a sliding window attention mechanism, thereby further diminishing computational complexity. Recognizing attention-based methods’ efficacy in emphasizing global information, we address the shortfall in modeling local information by introducing INN blocks. These blocks effectively preserve image information, enabling lossless transmission. To enhance fusion results, CC joints the loss function. Ultimately, the image reconstruction module combines a fused image based on both global information and local features. The whole approach aims to leverage the strengths of attention mechanisms and inversion, bridging the gap between local and global information modeling for superior image fusion outcomes. The results on MSRS, M3FD, and TNO datasets demonstrate our promising performance on IVF tasks.

With the increasing demand for underwater environment monitoring and remote object identification, underwater object detection algorithms based on forward-looking sonar images have become a research hotspot. A kind of algorithm called remote deep feature fusion detection network (RDFFRN) is proposed, which aims to solve the problems of object shadow and background confusion and low detection accuracy of small objects in the underwater object detection algorithm of sonar images. First, the model proposes the deep feature extraction and fusion module (DFEFM). It achieves more multilayered feature fusion and captures more local details. Finally, the RDFFRN proposes a new head detection network, which increases the proportion of small-sized feature maps and improves the model’s discrimination ability for small underwater objects. After extensive experiments on a sonar images dataset, the RDFFRN is verified to be superior to other object detection methods. The RDFFDN improves the mean average accuracy by 3.2% over the baseline model. It can be shown that RDFFRN has a broad application prospect in the underwater operation field.

Object detection in unmanned aerial vehicle (UAV) images has recently become a popular tool with widespread applications, but due to the high altitude of UAV and the subtle inter-class differences, effectively detecting object locations and achieving accurate classification remain a challenge. Existing methods often rely on multi-scale feature fusion and attention mechanisms to address these problems. However, these methods tend to significantly increase the computational overhead and introduce unwanted background noise interference in shallow features. To address these issues, we propose a tiny object detection network for aerial imagery based on a deformable cross-attention module and enhanced classifier module (DCENet). Specifically, a self-designed deformable cross-attention module makes shallow feature maps adaptively focus on regions of interest in deep feature maps, enhancing the detection of object locations without being affected by additional background noise interference. Meanwhile, a crop-images super-resolution module (CSRM) is used to enlarge the cropped images, solving the problem that tiny objects have lower resolution and appear more blurred compared with ground-view objects. In addition, an enhanced classifier module is adopted to enhance the model’s classification capability for some similar categories, thereby improving the overall performance of the network. The module integrates both the CSRM and the ResNet-34 classifier. The experimental results show that DCENet achieves state-of-the-art performance with mean average precision values of 36.0 and 27.5 on the VisDrone-2019 and UAVDT datasets, respectively. This suggests that the proposed DCENet is more suitable for aerial image detection.

Accurate detection of surface defects is crucial for maintaining the quality of strip steel products. We propose a lightweight strip steel surface defect detection model, YOLOv8-VRLG, to address the issues of insufficient detection accuracy and large parameter sizes in existing models. First, the VanillaNet module was integrated into YOLOv8 to preserve efficient feature extraction capabilities while effectively lightening the backbone network. Second, the C2fRL module, which integrates the benefits of RepGhost and large selective kernel modules into the original C2f, was introduced to further reduce the number of parameters and dynamically adjust processing details based on image content, thereby enhancing detection accuracy. Finally, the traditional convolutional layers in the detection head were replaced with the GSConv module. This module optimizes information exchange and feature fusion with its unique mixed convolution strategy, thereby improving the model’s accuracy and computational resource efficiency. Extensive tests on the NEU-DET dataset demonstrated that the mean average precision of the YOLOv8-VRLG model improved by 1.8% compared with that of the YOLOv8n model. In addition, the computational cost and parameter count were reduced by 23.5% and 28.6%, respectively. Tests on the GC10-DET dataset further showcased the model’s exceptional adaptability and robustness in complex environments.

LiDAR-based 3D object detection is crucial for perception systems in autonomous driving. However, current methods perform poorly in detecting occluded and small objects due to the sparse and uneven distribution of point clouds in 3D scenes. To address this problem, we propose a long-range and multi-scale information interaction network (LMIINet) for 3D object detection. First, a feature extraction backbone with a spatial feature pyramid block is designed to effectively capture long-range dependencies between features. Then, multi-scale spatial features from the 3D backbone are adaptively fused at the neck to aggregate local and global contextual information. Finally, by fully utilizing the correlations between bounding box parameters, the proposed rotation-decoupled corrected intersection-over-union (RCIoU) loss is employed for the classification and regression supervision of bounding boxes to improve the one-stage point cloud detectors. LMIINet achieves competitive performance on small objects (pedestrians and cyclists) detection in the KITTI dataset. Compared with the benchmark network SECOND, LMIINet increases mAP3D/mAPBEV by 1.98%/2.09%, 12.25%/11.14%, and 6.44%/6.45% for the car, pedestrian, and cyclist classes, respectively.

Video anomaly detection (VAD) aims to identify events and behaviors in video sequences that deviate from established normal patterns. Traditionally, unsupervised VAD has been viewed as the one-class classification (OCC) task, which is predicated on the use of training data that encompass only normal events and exclude any anomaly samples. This approach can recognize previously unseen anomaly patterns. However, it may misclassify unfamiliar normal patterns as anomalies. In addition, as anomalous samples are not included in OCC training, it leads to unclear classification boundaries, which reduces the generalization ability of the model. To address these challenges, we introduce the self-supervised memory-guided and attention feature fusion method, which models normal events using optimized memory modules and attention feature fusion modules. The method not only generates pseudo-normal and pseudo-anomaly data but also significantly enhances the capability of the model in key feature identification and exploitation, thereby improving the capture and generalization of complex data relationships. Experimental results from three benchmark datasets-UCSD Ped2, CUHK Avenue, and ShanghaiTech show that our method achieves AUROCs of 99.5%, 90.9%, and 81.8%, respectively, demonstrating the efficacy of our approach. Our code is available at https://github.com/jzt-dongli/Self-Supervised-Memory-guided-and-Attention-Feature-Fusion-for-Video-Anomaly-Detection.

Sparse unmixing plays an important role in hyperspectral image analysis due to its ability to sparsely estimate abundances with a potentially large-scale spectral library. However, most of the current unmixing techniques tend to run into bottlenecks due to their tendency to oversimplify the optimization problem by incorporating convex regularizations, which may not adequately address the intricacies present in complex data representations. To address this problem, we propose a nonconvex sparse unmixing method by employing a tensor low-rankness penalty, named NSUTLR. Specifically, the proposed NSUTLR considers ℓ1/2 regularization to enhance the sparsity of abundances, which further couples with the row-sparsity-promoting iterative weights. Besides, the proposed NSUTLR characterizes the low-rank property by applying a nonconvex tensor nuclear norm based on the log-sum of singular values. The proposed NSUTLR can be solved by applying the alternating direction method of the multiplier framework. Experiment results obtained from the proposed NSUTLR and competitors conducted on simulated and real hyperspectral images demonstrate the superior effectiveness of our proposed method.

We propose a image-to-painting translation method that can generate paintings on a stroke-by-stroke basis. Unlike previous pixel-to-pixel or sequential optimization methods, our method generates a set of physically meaningful stroke parameters, which are closer to the way humans draw. These parameters can be further rendered using a renderer. We add an attention mechanism network to the proposed renderer to improve the quality of the painting images and use smooth L1 loss in the training process of the renderer to make the model converge faster. Our method can join neural style transfer, and we used Visual Geometry Group perceptual loss in the neural style transfer stage to get more realistic results. The experimental results show that the renderer used in our method is better than other renderers, and peak signal-to-noise ratio evaluation metrics are improved by 4.9% compared with previous renderers.

Pansharpening aims to enhance the spatial resolution of low-resolution multi-spectral (LRMS) images by leveraging the high-resolution panchromatic images (PANs). Deep learning–based pansharpening technology provides a suite of sophisticated unsupervised algorithms. However, there are several challenges: (1) existing methods introduce more or less some kind of a priori information but lack explicit physical priori. (2) Existing methods often lack generalization capabilities and perform well only on some specific datasets. To address these issues, an unsupervised feature-aligned pansharpening framework based on a generative adversarial network (FaPGAN) is proposed. The proposed FaPGAN employs a single-generator dual-discriminator structure, divisible into the following two steps: (1) the generator adopts a pair of adaptive complementary normalization factors to weight the fusion of LRMS and PAN, which captures the successive transition from LRMS to PAN. The dual discriminator is responsible for preserving the spatial and spectral information. (2) A novel loss is introduced, which constrains the representations in the feature domain for feature alignment, essentially improving prediction accuracy. The experimental results show that the algorithm exhibits excellent spatial and spectral fidelity when fusing images from Gaofen-2 and WorldView-3 satellite datasets, and it can be well generalized to reduced-resolution images.

We proposed a “hybrid fusion multi-ResNet transformer” (HF-MRT) model to improve the diagnosis of atopic dermatitis (AD) by using both hand-crafted and deep learning features. The Vietnam Atopic Dermatitis (VNAD) dataset, consisting of 7861 images of AD and non-AD cases, was utilized. Preprocessing and data augmentation techniques were applied to enhance model performance and generalization. Experiments were conducted across three feature scenarios: the first scenario focused on hand-crafted features, including a histogram of oriented gradients, local binary pattern, and color coherence vector; the second scenario involved deep learning features extracted from ResNet models (ResNet-50, ResNet-101, and ResNet-152); and the third scenario combined both hand-crafted and deep learning features. In all scenarios, high sensitivity rates were achieved by the HF-MRT model, reaching 91.4% in scenario 1, 93% in scenario 2, and 93% in scenario 3, providing reliable identification of most AD cases. The specificity rates, however, differed: 40.2% in scenario 1, 56.2% in scenario 2, and 60.9% in scenario 3. Scenario 3, with a sensitivity of 93% and a specificity of 60.9%, demonstrated diagnostic potential for detecting AD. Nevertheless, improvements in specificity remain necessary to enhance clinical applicability, ensuring accurate discrimination between AD and non-AD cases.