This PDF file contains the front matter associated with SPIE Proceedings Volume 11878 including the Title Page, Copyright information, and Table of Contents.

Video analytic techniques have been used to extract high level information from video streams. The technique leverages advances on machine learning to summarize complex image data into simple alert-signal to attract the attention of human operators. For example, in a station for the underground video analytic can help the operator to focus on an event from a specific camera rather than leaving this only to the human eye. A concern of such techniques is privacy as they expose people identity and enable profiling of personal habits and orientations. This work introduces ReSPEcT (Privacy Respecting theRmal basEd Specific Person rECogniTion), a privacy preserving video analytic system based on thermal video streams. ReSPEcT is able to identify a specific-human in thermal video streams from low-cost, low resolution cameras. The system leverages recent advances in machine learning (CNNs) and a plethora of pre-processing mechanisms, such as image automatic labeling, image segmentation, and image augmentation to reduce the stream background noise, improve resilience, strengthen human-body classification, and finally enable a specific human-target identification. ReSPEcT’s automatic labeling tool significantly reduces time thus automatically performing labeling using a model that can be retrained by an interactive web application. The experimental evaluation shows that overall ReSPEcT achieve 96.83% accuracy in identifying a specific person. Furthermore, is important to notice that while ReSPEcT can identify a specific human, the tool is not aware of the realidentity as it operates only on thermal images. ReSPEcT paves the way to use video analytic in a variety of privacyprotected scenarios, such as confidential meetings, sensitive spaces, or even public toilets.

New deep-learning architectures are created every year, achieving state-of-the-art results in image recognition and leading to the belief that, in a few years, complex tasks such as sign language translation will be considerably easier, serving as a communication tool for the hearing-impaired community. On the other hand, these algorithms still need a lot of data to be trained and the dataset creation process is expensive, time-consuming, and slow. Thereby, this work aims to investigate techniques of digital image processing and machine learning that can be used to create a sign language dataset effectively. We argue about data acquisition, such as the frames per second rate to capture or subsample the videos, the background type, preprocessing, and data augmentation, using convolutional neural networks and object detection to create an image classifier and comparing the results based on statistical tests. Different datasets were created to test the hypotheses, containing 14 words used daily and recorded by different smartphones in the RGB color system. We achieved an accuracy of 96.38% on the test set and 81.36% on the validation set containing more challenging conditions, showing that 30 FPS is the best frame rate subsample to train the classifier, geometric transformations work better than intensity transformations, and artificial background creation is not effective to model generalization. These trade-offs should be considered in future work as a cost-benefit guideline between computational cost and accuracy gain when creating a dataset and training a sign recognition model.

In this paper, hybrid network based on convolutional neural network (CNN) and long short-term memory network (LSTM) is proposed to improve hand gesture recognition accuracy. Distinguish from the large number of traditional surface electromyography (sEMG) features proposed by previous researchers, without involving a lot of manual design and professional domain knowledge, this hybrid network can automatically extract both spatial features and temporal features from the input sEMG signals. The hybrid CNN-LSTM Network has two parallel feature extraction stages: spatial features extraction using CNN and temporal features extraction using LSTM. The hybrid CNN-LSTM network combines spatial features and temporal features as Hybrid features (HybridFeat) and feeds HybridFeat into traditional classifiers, including linear discriminant analysis (LDA), support vector machine (SVM) and K nearest neighbor (KNN). The experiments showed that both in inter-session scenario and inter-subject scenario, the HybridFeat outperforms all the tested traditional features and CNNFeat. Besides, it was found that combining HybridFeat with traditional features can further improve the accuracy.

Purpose: The current study aims to investigate the effect of orofacial motions for automatic attitude recognition Method: To achieve this goal, the movements of lips and jaw during the expressions of six common attitudes from 33 native Mandarin Chinese speakers were collected using ElectroMegnatic Articulography. The random forest classifications were then conducted for attitude recognition based on orofacial data. Results: the average rate of attitude recognition was 63.45%, and the identification accuracy was above 60% for almost every attitude. Besides, for the further classifications separately conducting on each pair of attitudes, the opposing attitudes within each attitude pair were all reasonably recognized (i.e., above 65%). Conclusion: The use of orofacial expressions in attitude expression could be valuable features for the technology of automatic attitude recognition.

Historical documents record a lot of precious information through ancient characters. However, some problems like unbalanced character samples and intra-class multi-modality inside the documents are critical factors that limit the performance of existing character recognition technologies. Therefore, we propose a two-stage font generation model, Blend-Font Net, which use some easy to get modern character datasets to augment ancient character dataset and solve these mentioned problems based on blend-font strategy. The model generates new samples by extracting and modifying the font information from the character image. A font generation model learns the mapping between different fonts in the first stage, and the slightly modified model learns how to generate samples that blend two different fonts in the second stage. Extra samples are generated for balancing historical documents dataset through the proposed model. Experiments show that our results have diverse visual effects and improve the accuracy of the text recognition network. Furthermore, the proposed method shows a broad application prospect in similar works as no font label required and multi-modality problem solved.

Person recognition over time is a bit challenging task as compared to re-identification in multi-camera environment. Usually, people appear after certain time period at public places like airports, carrying accessories and changing of clothes etc. In this paper, we proposed a newer recognition framework using two types of images i.e. whole and upper body silhouette. A customized version of DeepLabv3 is used for human body semantic segmentation. The Generic Fourier Descriptor (GFD) based feature set is fed to One-Vs-Rest schema in ensemble of K-Nearest Neighbor (KNN) and Random Forest (RF) classifiers. The experiments are carried out on Front-View Gait (FVG) dataset recorded in year 2017 and 2018 respectively. An overall recognition accuracy of more than 93% is noted using both classifiers on whole body human silhouette images. On the other hand, upper half human silhouette obtained recognition accuracy of more than 91% and 88% using RF and KNN respectively. Code is available at https://git.io/JtfMY

Inspection of glass façade and concrete structures for damages such as cracks are crucial for buildings safety and maintenance. Such surface cracks result in appearance of objects of abnormal geometric shapes, an obvious motivation to investigate geometric descriptor features associated with visible objects on surfaces of such materials. Here, we are concerned with developing generic automatic vision-based inspection for detection and recognition of such infrastructure anomalies. This paper extends and enhances our earlier work on glass façade cracks. We propose and test the performance of several handcrafted texture feature descriptors to discriminate cracked surface material. These features include the Histogram of Oriented Gradients (HOG), the Uniform Local Binary Pattern (ULBP), together with quantised Linearity and curvature measures obtained post an edge detection procedure. Unlike the previous paper, we extract and concatenate these features from a 3x3 blocks that partition the input image. The performance of the proposed methods is tested on four datasets of glass cracks and a large dataset of concrete cracks. We shall demonstrate that the block-based approach yields significant (5%-10%) improvement compared to our earlier work on glass, and all features have high performance for concrete surfaces attaining 98.6% for HOG feature. Furthermore, we adapt CNN layers trained on the ImageNet dataset and transfer this knowledge to the crack surface recognition task. The significant efficiency of handcrafted features, compared to CNN models, raises issues on models suitability for implementation on board UAV and constrained mobile devices.

A person's signature is a significant biometric trait that can be used to identify the person, often required in financial transactions and insurance-related activities. It is well-known that during such transactions, handwritten forgery signatures made on behalf of a genuine user results stealing the money or other means of wealth which was safely kept in the bank or other institutions. Therefore, it is necessary to have a vital safeguard like an automated signature verification system against malicious offenders who threaten society. This paper reports a writer independent offline signature verification system that makes use of genuine and forged signatures written in Manipuri script. Further, a combination of handcrafted geometric features and the features extracted using Convolutional Neural Network (CNN) is used. Then, the combined features' feature space is made optimal using the Genetic Algorithm (GA). This system has achieved a very high-level performance using an ensemble of four pattern classifiers, Support Vector Machine (SVM), k-Nearest Neighbours (KNN), Naive Bayes learning, and Decision Tree Learning. Ensembling of the classifiers is done using logical OR rule and Majority Voting. Experiments are conducted on an original database consisting of Manipuri signatures of 81 individuals. Experimental results are compelling, while the proposed offline signature verification system is compared with the existing system.

In this work, we propose a novel personalized facial attractiveness prediction method that is able to effectively learn an individual’s preferences on few training images. A deep convolutional neural network (CNN) was first employed to estimate facial attributes. Then the attributes that play the most significant role for the individual were selected to train Random Forest. A new dataset specially created for personalized beauty evaluation was also proposed. Our method has achieved promising results of 54% Pearson’s Correlation on 15 training images, 61% on 35 images.

Image captioning involves generating a sentence that describes an image. More recently, it has been driven by encoderdecoder approaches where the encoder such as convolutional neural network (CNN) can extract the visual features of an image. The extracted visual features are passed to a decoder such as a long short-term memory (LSTM) network in order to generate a sentence that describes the image. One major challenge with this approach is to precisely include the scene of an image in the generated sentences. To resolve this challenge, visual scene features have been used with unidirectional LSTM decoders. However, for long sentences, this limits the precision of the generated text. This research proposes a novel approach to generate sentences using visual scene information with a bidirectional LSTM decoder. The encoder is based on Inception v3 to extract the object features and Places365 to extract the scene features. The decoder uses a bidirectional LSTM to generate a sentence. The encoder-decoder model is trained using the Flickr8k dataset. Results show improved performance for generating longer sentences with a 9% increase in BLEU-3 and a 12% increase in BLEU-4 scores compared to compared to other encoder-decoder methods that are limited to only using global image features. Visually impaired people that use screen readers would benefit from this research as they would get an enhanced description of an image that includes the background scene thereby creating a wholesome picture in the mind of the reader.

The SIFT (Scale Invariant Feature Transform) feature is a descriptor of the key points in the gray-gradient spatial. It cannot distinguish samples with the same SIFT features and the varied color distributions. The paper proposes a novel feature expression, named HSH-SIFT, which firstly extracts the SIFT feature of Value channel in HSV image, secondly calculates the hue-saturation histogram (abbrev. HSH) of the neighborhood of the key point and flatten it to be a 96-D vector, finally attaches the HSH vector to the tail of the corresponding SIFT feature vector to construct a 224-D fusion feature vector. Secondly, it proposes a fusion match strategy to improve the precision of content-based image retrieval, it fuses k-D tree, NNDR strategy, unified perspective transformation, and symmetric test to filter mismatched feature points. Experiments have shown that it significantly effects the precision of ceramic image retrieval and has been applied to ceramic image retrieval application.

This paper proposes a novel method to utilize three source features for video captioning. It fuses global video features with local object and regional features to model the relationships among objects and their motions and applies object tags instead of visual features to guide the generation of descriptions. Specifically, Multi-feature is firstly extracted by pretrained models and treated as separate inputs alongside video frames. Secondly, an object awareness attention block is designed to fuse the different features information and to learn a joint video representation which has both visual and linguistic semantics. Experiments on MSVD and MSR-VTT datasets have shown the effectiveness of the proposed method, and the ablation studies have verified the contribution of each component.

Research on object detection algorithms with higher accuracy and faster detection speed is currently the main concern. In order to improve detection performance, an improved object detection algorithm using YOLOv3-tiny based on pyramid pooling is proposed. First, an improved pyramid pooling module using adaptive average pooling is designed to efficiently extract global feature information, and then combine the module with YOLOv3-tiny to explore the impact of different combinations on the detection results. The experiment used PASCAL VOC2007 trainval and all PASCAL VOC2012 for training and validation, and used PASCAL VOC2007 test for testing. Experimental results show that the proposed network improves mAP by 1.8% compared to YOLOv3-tiny while the detection speed is almost the same, which better achieves the balance of detection speed and accuracy.

Pulmonary nodule detection system consists of two steps: candidate detection and false positive reduction. To dynamically adapt the sizes and ratios of the nodules, Local Density based Iterative Self-Organizing Data Analysis Techniques Algorithm (D-ISODATA) is proposed for automated anchor boxes configuration. For candidate detection, instead of fixed anchor, D-ISODATA is utilized for automatically generate anchors to adapt to high variability of nodules. D-ISODATA initializes clustering center and removes noises based on the principle of maximum local density and further clustering is carried out with self-adaptability. In addition, attention mechanism is adopted in feature channels to enable the model to focus on nodule-related features. For false positive reduction, 3D Resnet is utilized to extract the three-dimensional features of nodules. Experiments are carried out on LUNA16 dataset and show out a sensitivity of 93.6% with 0.15 false positive per scan. The results show preferable performance of the proposed method.

The angle of steady arm is an important inspection parameter of catenary. The existing manual measurement methods were unable to meet the requirements on measurement efficiency or measurement accuracy, which greatly restrict the efficiency of defects inspection and maintenance of catenary on high-speed railways. An automatic visual measurement method with excellent measurement efficiency was developed for the angle of steady arm, which can be used on catenary inspection car. However, the inspection reliability is poor for images with complicated background, such as images at the posts with multiple cantilever and between two posts with no steady arms. In order to solve the problem on system reliability, an upgraded visual measurement method is proposed in the paper. The camera system is changed into trigger image acquisition mode with a post detection module integrated, and the image detection algorithm for steady arm is greatly improved using deep convolutional neural networks, exploiting the research progress on object detection for catenary component. The proposed system has been fully tested on detection reliability, measurement repeatability and measurement accuracy, which shows much better reliability and availability.

Computer vision is a field of artificial intelligence that trains computers to gain high-level understanding from images or videos. Among the most well-known subfields in Computer Vision are object detection, object tracking, motion estimation, etc. So, object detection is a computer vision technique for detecting instances of objects in images or videos. It can be performed in different ways, either by using delimiting frames or by using object segmentation. It is useful and widely used in the following tasks: image annotation, activity recognition, face recognition, and object tracking. There are two types of approaches to detecting objects in the image: two-stage detector-based approaches and one-stage detector-based approaches with their advantages and disadvantages. Object detection algorithms and techniques generally fall under machine learning approaches and deep learning approaches. Deep learning approaches are based on convolutional neural networks that allow us to perform many tasks, such as clustering, classification, or regression. This paper, which is a survey, aims at reviewing the different approaches and models for deep learning to detect objects in the image and then the advantages and disadvantages of each approach as well as the different fields of application. Also, many data sets that are proposed to evaluate the different methods to detect objects in the image will be presented in this article.

Multiple target tracking (MTT) or multiple object tracking (MOT), which is a key stage operation for many computer vision applications, has relied and it relies on detecting and identifying targets within videos. The objects of these videos may be, for instance, pedestrians, vehicles or animals regardless of the number of targets or their appearance. The main goal of multi object tracking is to follow paths or trajectories of multiple targets in a sequence. Multiple object tracking is a field of computer vision that has a wide range of applications, starting from video surveillance and human computer interaction to robotics. The aim of visual object tracking is to maintain objects’ identities. Developing an effective and accurate visual tracking system is very challenging due to the enormous number of problems, such as Illumination variation or background, the famous Clutters meaning the overlap between objects. Low resolution may be due to cameras problems, scale variation, occlusion, and change of targets’ positions within video. These challenges have a wide range of approaches and methods in this evolving research area with several algorithms in each subfield. This paper covers the crucial research area for Multiple Object Tracking (MOT) and this study will help researchers accomplish their scientific projects relying on the wide range of algorithms mentioned on this review.

With the development of internet technology, artificial intelligence shows its rapid growth in recent year as well. More and more people pay attention to this field, including criminals inevitably. It is noted that one technology called “Deepfake” appeared on the internet at the end of 2017. As the name suggests, it is a portmanteau of “deep learning” and “fake”. In essence, Deepfake is a deep-learning framework in the field of image composite and replacement, which swap faces in images or videos. However detection of face forgery is still in its early stages, due to its novelty and complexity. In this paper we will demonstrate some dataset for deepfake forensics firstly, and then describe various existing detection methods. Those methods are reviewed in two parts: detection based on frame sequences and detection based on single frame. The former is implemented by differences between frames including human features, optical flow, timeline and so on, while the latter is based on features of single frame including extracted features and fusion boundary. Various convolutional Neural Networks (abbreviated as CNN) will be illustrated in this paper. Accordingly, performances of above algorithms are likely to be demonstrated and compared, and a further explanation will be given regarding on their applicable dataset. Finally, further research of face forgery detection of deepfake including methods and applications will be discussed.

Due to the increasing demand for deploying CNNs on resource-constrained platforms, efficient neural networks are becoming more and more popular, in which depthwise convolution plays an indispensable role. Recently, larger kernel sizes (≥5) have been applied to depthwise convolution, but with significantly increased computational cost and parameter size. In this paper, we propose a novel extremely separated convolutional block (XSepConv), which fuses spatially separable convolutions into depthwise convolution to substantially reduce both the computational cost and the parameter size induced by large kernels. We also propose an extra 2×2 depthwise convolution coupled with a new and improved symmetric padding strategy to compensate for the side effect brought by spatially separable convolutions. XSepConv is a more efficient alternative to vanilla depthwise convolution with large kernel sizes; moreover, extensive experiments on popular image classification and object detection benchmarks demonstrate that XSepConv can strike a better trade-off between accuracy and efficiency, and the improvement is more significant for larger kernels.

Pest detection is a critical issue for farmers and pest analysts and requires immediate response to control the damage. However, one of the current pest control evaluation strategies is to manually mark the pests from the sample image and count them, which is tedious and time-consuming. This paper presents a method to automatically detect and count the number of small yellow thrips (SYT) on lotus leaf back based on YOLO. Due to the natural curved structure of the leaf, some parts of its picture will be out of focus and therefore result in inaccurate counting results. In order to reduce the impact of the out-of-focus area, we first proposed a novel SYT-simulation experiment and then trained the Very-Deep Super-Resolution (VDSR) and Deep plug-and-play super-resolution (DPSR) neural network to enhance the blurred area. With such image enhancement, the final detection rate can be improved from 70.41% to 80.17% after passing a YoLobased object detection network.

Deep learning is increasingly used in every medical imaging segmentation task, but detection of lesions from eye fundus images (EFI) poses many difficult challenges related to sizes, similarity with other lesions and structures, low contrasts, variant conformations. During training, the loss function directs backpropagation learning in the deep convolutional neural networks (DCNN) that are used. It is therefore a fundamental function to the optimization procedure. There exist alternative formulations, such as cross entropy, jaccard and dice. But does the choice of loss influence quality decisively, in the difficult context of EFI lesions? And what about the network architecture? As part of our effort to improve the approaches, we evaluate alternative loss functions, also alternative architectures. We show that the choice of a convenient architecture and loss function can double the quality detecting some of the small and difficult to detect lesions, but we also show that research is still required to find ways to improve the results further.

Semantic segmentation is an intensive prediction task with a wide range of applications, such as mobile phones, robots, autonomous driving and other mobile devices. The computing power of these mobile devices is often very limited, which limits the application of semantic segmentation. This paper designs a simple and efficient end-to-end real-time segmentation algorithm. Our algorithm model is an asymmetric encoder-decoder structure, which is composed of two parts: the first part is a lightweight encoder, which consists of a dual-path fast downsampling module and a lightweight three-stage feature extractor; the second part is an asymmetric decoding classifier, which consists of a mutual attention feature fusion module, a lightweight atrous spatial pyramid pooling module, and a classifier. Our method has only 0.64M parameters, and achieves 68.4% mIoU, 67.5 FPS with an image of 10242048 resolutions, also reachs 203.8 FPS, 61.9% mIoU with 5121024 resolutions, on the Cityscapes test set on GTX1080Ti. Comprehensive experiments show that our method can reach the state-of-the-art level on the Cityscapes dataset in terms of accuracy and speed.

Semantic segmentation with deep learning has achieved remarkable progress in classifying the pixels in the image. Acquiring sufficient ground-truth supervision to train deep visual models has been a bottleneck over the years due to the data-hungry nature of deep learning. Image-level label-based weakly supervised semantic segmentation (WSSS) aims to adopt image-level labels to train semantic segmentation models, saving vast human labors for costly pixel-level annotations. A typical pipeline for this problem is to adopt Class Activation Maps (CAMs) with image-level labels to generate pseudo-masks (a.k.a. seeds) and then use them for training segmentation models. The main difficulty is that seeds are usually sparse and incomplete. In recent years, GCNs have made great strides in various fields. GCN can perform global modeling and reasoning on the relationship between regions, which is beneficial for many computer vision tasks. This is our motivation to combine these two aspects. Therefore we propose the module of class-related graph convolution. Because there are differences between classes, our GCN is parallel. Each GCN can learn the classrelated region extension strategy. To enable GCN to learn more authentic relationships, we also introduce the attention mechanisms. We conduct lots of experiments on the public PASCAL VOC dataset, and our model yields state-of-the-art performance.

Aiming at the problem that SAR image segmentation is seriously interfered by the speckle, this paper studies the methods of SAR image segmentation based on different gradients under nonlocal mean (NLM) filter. We focus on four gradient operators, contained ratio of average(ROA), ratio of an exponentially weighted averages (ROEWA), Unsampled Wavelet Transform (UWT) and instantaneous coefficient of variation (ICOV). Firstly, the NLM filter is used to suppress the speckle and protect the structure in the SAR image. After that, the gradient of SAR image is calculated based on the above different gradient operators. Finally, the watershed technique is used to segment the image based on the gradient of SAR image. Experimental results show that the segmentation based on ICOV and UWT obtain more accurate boundary location and fewer segmentation regions than the methods based on ROA and ROEWA.

In this work, we proposed a method combined the fuzzy spatial correlation of voxels in the MRI images obtained from a 3D network using CRF with the slice information captured by an ordinary 2D network to focus on the brain tumor segmentation task. Considering the expensive devices required by 3D networks while 2D networks can loss the information in the channel direction which leads to many false positive predictions, the proposed one can be a favorable direction to get more accurate features of the brain tumor. We take MRI images with 4 modalities in BRATS2018 dataset as the input of the 3D CNN after reducing the resolution. The CRF is used to calculate the neighboring correlation after the CNN feature extractor and can generate the probability map. The 2D network takes 2D slices in 4 modalities from the MRI images as input and output the segmentation map. The 2D segmentation maps are joining to 3D in order and combined with the probability map to get the final result. Compared with the state-of-the-art and the baseline method with the average Dice less than 0.85, the proposed is time and memory saving with the average Dice nearly 0.88.

Nucleus segmentation is a fundamental prerequisite in the digital pathology analysis. However, automated nucleus segmentation is challenging due to clustered arrangement and possible occlusion. Additionally, some nuclei exhibit large variability between images and have fuzzy boundaries. Most of the previous works solve the task through FCN, which requires well-designed post-processing methods to separate instances. In contrast, Mask R-CNN segments objects based on region proposal with no post-processing methods to separate instances, but usually confuses the foreground and background. In this paper, we propose a Semantic Feature Refine Module (SFRM) to enhance its ability to distinguish foreground and background. We first add a semantic segmentation branch to enhance the semantic feature of FPN. Besides, we utilize the feature of semantic segmentation branch to yield an attention pyramid for FPN to enhance its semantic feature further at the same time. Experiments on CoNSeP and PanNuke datasets verify the effectiveness of our method.

Road segmentation is an important part of autonomous driving vehicles. Reliable road segmentation results are a prerequisite for autonomous driving tasks, e.g., path planning. In this paper, we propose a road segmentation method based on 3D point cloud organization and line scanning of LiDAR data. Our model assumes that road areas are always flatter than non-road areas. First, we propose an adjacent-line-difference (ALD) feature to define the flatness of the point cloud. Using this feature, the approximate road area can be estimated. Then, we proposed a horizontal and vertical scanning strategy to obtain a more accurate road area. In order to prove the effectiveness of our method, experiments are conducted on the public KITTI-Road benchmark. The experimental results prove that the proposed approach can achieve one of the best performances among all LiDAR-based methods.

Aiming at the problem of inaccurate classification of textures at different scales in traditional texture classification methods, this paper proposed a deep convolutional neural network (CNN) based on improved residual blocks to increase the accuracy of texture classification. First, the two convolution layers in the original residual block were replaced with two dilated convolution layers, and a spatial attention module after the second dilated convolution layers was inserted. Thereafter, the residual connections were used for feature fusion to obtain a greater receptive field and attention-enhanced features. Second, based on the improved residual blocks, a multi-scale texture classification CNN was stacked in a way of increasing the number of block channels. The experiment was performed on a 64-class texture dataset. Experiments show that, compared with the state-of-the-art methods, the proposed method achieved a higher classification accuracy of 99.17%.

Recognize fine grained categories is challenging task, which has attracted more and more attention in recent years. Different from traditional image recognition, fine-grained image recognition is to recognize different sub-classes under the same general category. Due to posture, illumination and other reasons, fine grained recognition has large intra-class diversities and subtle inter-class similarities, Most of the works focused on how to localize discriminative regions and fine-grained feature learning. But they negative the structure of the fine grained labels. In this paper, we propose label hierarchy constraint network (LHC) for fine-grained classification. The network include two branchs, coarse-level branch and fine level branch. In the middle layers of the neural networks, we use the coarse branch to predict the coarse labels, which can be regarded as the guidance of fine-grained labels. In the upper layers, we predict fine-grained labels. Then we map the result of coarse-grained branch prediction probability distribution to the fine-grained branch. Experiments show the effectiveness of our method.

In the paper, the classification capabilities of perceptron and radial neural networks are compared the example process identification of graphical compost quality. The classification was based on graphical information coded as selected quality features of the compost quality, presented in colour digital images. In the paper, MLP (MultiLayer Perceptrons) and RBF (Radial Basis Function) neural classification models are compared, generated using learning sets acquired on the basis of information contained in digital photographs of compost. In order to classify the compost pictures, modern neural modelling methods were used, including digital image analysis techniques. The qualitative analysis of the neural models enabled the compare of the MLP and RBF neuron topology and identification ANN that was characterised by the highest classification capability. Characteristic features enabling effective identification of a pest were 16 selected graphical parameters. The created neuron model is dedicated as a core for computer systems supporting decision processes occurring during compost production.

Few-shot learning aims to learn a classifier with limited training instances to recognize unseen classes in test. Recently, some effective few-shot learning approaches have achieved promising classification performance. However, these approaches do not pay attention to the importance of task-relevant features in image classification. To address this issue, we propose Task-Relevant Graph Metric Learning method that adopts a novel meta-learning framework for transductive inference. Specifically, we first extract task-relevant features by a Squeeze-and-Excitation module. Then learning a graph construction module in order to obtain the manifold structure in the data. Afterward, self-training is utilized to propagate labels from labeled instances to unlabeled test instances. Experiments on benchmark datasets demonstrate that TRGML improves classification performance (4%-5%) over baseline systems on miniImageNet and tieredImageNet.

Current CNN-based deep feature learning methods for image classification are mainly focused on learning with two types of loss functions: the softmax loss and its variants with class-level labels, or the metric-learning losses with pairwise labels. The former regards the weight vectors of the last fully connected layer of CNNs as representations of the class centres and focuses on the similarity between a single feature and each weight vector. The latter aims at seeking appropriate distance metrics between features pairs to boost the CNNs by learning a discriminative embedding space. In this work, we propose a simple yet effective new loss function called angular joint loss (AJL) to exploit the best of both worlds. By integrating feature-weight angles with feature-feature angles, AJL inherits the simplicity of softmax loss and the power of pair-wise metric learning and enhances both the intra-class compactness and the inter-class separability. The proposed AJL is simple to implement and has no time-consuming mining of pair or triple samples. On some popular image classification datasets (CIFAR-10, CIFAR-100 and ImageNet), extensive experimental results show that the proposed AJL can achieve the state-of-the-art performance.

Convolutional Neural Networks (CNNs) are widely used in image classification tasks and have demonstrated promising classification accuracy results. Designing a CNN architecture requires a manual adjustment of parameters through a series of experiments as well as sufficient knowledge both in the problem domain and CNN architecture design. Therefore, it is extremely difficult for users without prior experience to design a promising CNN for their purposes. To solve this issue, various solutions on the automatic construction of CNN architectures are proposed, including but not limited to a genetic algorithm AE-CNN 1 for automatically evolving CNN design using ResNet and DenseNet blocks. In this paper, a significant improvement of the afore-mentioned solution by the addition of Inception blocks is introduced. Performance of the algorithm is assessed on the CINIC-10 benchmark dataset without and with the usage of Inception blocks. As it can be observed from the outcome of the experiment, the addition of Inception blocks positively affects the final classification accuracy. The proposed algorithm does not only improve the current solution but also keeps the advantages of automatic CNN without requesting any manual interventions.

Due to traditional classification methods can’t meet the requirements of a great variety of cardiovascular diseases in clinical environment, CNN recognition method based on wavelet denoising is proposed to apply in automatic analysis of multi-category heart sounds. Firstly, original heart sound signal is denoising by wavelet threshold and the Mel-Frequency Spectrum Coefficients are extracted as the features of heart sound. Then models of CNN and Long Short-Term Memory (LSTM) networks are constructed and the related parameters are adjusted. Finally, the trained models are used to classify the heart sound signals. The experimental results show all performance indicators including sensitivity of proposed algorithm outperform LSTM and the traditional methods, achieves a better recognition results at the same time.

In order to authenticate the content of the image and find out the tampered location, semi-fragile watermark technology usually extracts feature information from image blocks as watermark, which will be embedded into irrelevant blocks by block mapping algorithm. When a block is tampered, its original feature hidden in other blocks will not be changed. However, it must be pointed out that this situation only considers that the original feature information of the tampered block has been tampered, but ignores that the feature information of other blocks embedded in the tampered position is also destroyed. It is impossible to recognize that whether the tampered area itself or the image block whose feature information embedded in the tampered area is tampered, resulting in increasing of false detection rate. In order to improve the accuracy of tamper detection of semi-fragile watermark algorithm, this paper proposes a novel tampering detection algorithm, which performs multi-scrambling encoding on the extracted feature information, and establishing a tampering equation related to each block according to the multi-scrambling encoding. According to the sum of tampering equation to determine whether the image block has been tampered. Experimental results show that after using the tampering detection algorithm proposed by this paper, both the false rejection probability and false acceptance probability are lower than the previous algorithms. In addition, the false acceptance probability is still very low under JPEG compression.

Existing methods for remote sensing image denoising typically suffer from a common drawback of fuzzy edge information. In this paper, we proposed a Generative Adversarial Network(GAN) based on the residual learning and perceptual loss for image denoising. The proposed GAN is designed with the two parts: The generator network takes the high-frequency layer of noisy image as the input and outputs a clean image after training. In order to eliminate noise better while retaining more edges and details, three residual blocks are embedded in the generator and a perceptual loss function is added to learn the perceptual differences between the denoised images and the ground truth images. The discriminator network based on 70×70 PatchGAN can discern between the denoised image and the clean image through a confidence value. The experiments show that our proposed network achieves superior performances and preserve majority the edge contours and fine details from low-quality observations.

Image tamper detection techniques have been proposed to address the potential tampering problems of image content, as digital images can be easily modified, diffused, transformed and duplicated via the Internet. In order to achieve high detection successful rate and self-recovery accuracy in image tamper areas, an efficient scheme of rehashing model based digital watermarking for image tamper detection and self-recovery is proposed in this study. The rehashing model consists of a series of random hash functions, which is presented to avoid numerous random number collisions that embrace the authentication data for image tamper detection and self-recovery. Additionally, these authentication data can be further embedded into the original image as digital watermarking by means of the constructed two-dimensional reference matrix. The experimental results demonstrate that this novel scheme outperforms the other related works in terms of tamper area localization and self-recovery accuracy; moreover, the proposed image tamper detection and self-recovery technique can obtain an acceptable recovery image visual quality with half image content tamper.

The image denoising methods based on convolutional neural network can remove the noise better. However, some detailed features will be removed along with the noise, resulting in the image blurred. In this paper, we propose a denoising network based on edge enhancement and residual learning to make it generate higher quality denoised results. First, we use the Non-local Means (NLM) denoising algorithm to pre-denoise the noisy image, and then use canny edge detection on this result to obtain the corresponding edge matrix; Second, perform non-subsampled contourlet transform (NSCT) decomposition on the noisy image to obtain high-frequency sub-bands; Third, locate and amplify the edge coefficients of the high-frequency sub-bands through the edge matrix to obtain the edge-enhanced noise image; Finally, the edge-enhanced noise image is input into the residual denoising network training to get the final denoised result. The experiments show that when compared with the traditional denoising methods, our proposed method can remove more noise while retaining more rich edge information and have a better improvement in visual quality.

A robust image watermarking method based on neural network is proposed for websites, such as the social networks (Facebook, Twitter), which adopts different JPEG recompression strategies for uploaded images. The basic network structure consists of three parts: the encoder inputs a cover image and watermark message, outputs the watermarked image and realizes watermark embedding; the decoder inputs a watermarked image, outputs extracted watermark message; the discriminator and the encoder constitute a generating antagonistic network, which promotes the encoder to generate watermarked images which are inseparable from the cover image. A noise layer is designed in the network structure to simulate the attacks that watermarking may be subjected to, and the whole network is trained end-to-end to make the watermarking model robust to the attacks. The experimental results show that the noise layer is designed to simulate JPEG recompression process reasonably, and the model is robust to different JPEG recompression processes on Facebook and Twitter, with the average bit error rate less than 1%. In fact, embedding watermark adds noise to the cover image; considering the influence of the counter sample on the deep neural network, this paper verifies the interference of embedding watermark to the target classification network. By adding the target classification module into the original network structure, the influence of watermark embedding on the target classification network is considered at the same time. The experimental results show that the target classification module is helpful to reduce the interference of watermark embedding on various target classification networks.

A new method based on single shot using color coded image sequence for 3D reconstruction is proposed. The previous spatial neighborhood coding methods often need to determine the center of the pattern, but inaccurate positioning of pattern feature points will bring errors to 3D reconstruction. In this paper, a color-coded pattern is designed as two opposite triangular patterns to directly establish the center point of the pattern, thus avoiding the error caused by the inaccurate positioning of the center point after the pattern distortion. We designed a corner detector for our feature point detection, it is efficient and accurate. The pattern is coded in four colors of red, blue, green and pale yellow. We use a quaternary sixth order primitive polynomial to generate the pseudorandom sequence, which is then regenerated into the M-array. The final size of the code pattern is 65×63, and the unique code window size is 2×3. Experiments show that the 3D reconstruction owns high measurement precision and the decoding has strong robustness.

The Covid-19 is a highly contagious and virulent disease caused by the Severe Acute Respiratory Syndrome - Corona Virus – 2 (SARS-CoV-2). Over 146 million cases and 3.1 million deaths were reported worldwide as of April 27, 2021. A multinational consensus from the Fleischner Society reported that Computerized Tomography (CT) can be utilized for the early diagnosis of Covid-19. However, this classification involves radiologists’ time and efforts significantly. It is crucial to develop an automated analysis of CT images to save their time and efforts. In this paper, we propose CoviNet, a deep three-dimensional convolutional neural network (3D-CNN) to diagnose Covid-19 from CT images. We trained and tested the proposed CoviNet using two public datasets with radiologist-labeled CT images. The experimental results show the proposed CoviNet is promising.

Beam hardening caused cupping and light or dark streaks are one of the major problems in computer tomographic imaging. There are many possibilities to manage beam hardening artifacts. Deep learning is a set of techniques that proved to be useful in various fields, and it is more and more commonly used in computer tomography as well. Our goal in this paper was to reduce beam hardening artifact using deep convolutional neural networks and to develop a model, which can improve the quality of the beam hardening distorted image with the presence of high amount of electrical noise. Moreover, we investigated various ways of fusing deep neural networks with the process of computer tomographic imaging. For these purposes we produced a large heterogeneous simulated dataset. We tested the performance of four convolutional networks-based methods for correcting the distorted data at various stages of the tomographic imaging pipeline. The evaluations were based on three different error measures. The evaluations have shown that the best result can be achieved by the method which makes the reconstructions step as a part a neural network. This method was capable of removing a significant amount of distortion in case of high level of electrical noise.

Technical Photography (TP) is a non-invasive technique used for analyzing an artwork to decide the conservation method, pigment classification, hidden detail revealing, and painting retouching. This technique requires a modified digital camera that removes a visible-pass filter to acquiring a set of broadband spectral image. The problem with this technique is that it takes time during the image acquiring process due to several combinations of light source and filter in order to collect all information in the painting. Our research aimed to develop a system that can reduce time in the image acquiring process by using one digital camera and one modified digital camera to acquired two images in different spectral bands simultaneously. All acquired images are processed with image processing software. Full-frame DSLR (Nikon D850, 45.7 megapixels) and a mirrorless (Nikon J5, 20.8 megapixels) camera are compared in the topic of image register, image quality and the performance were analyzed in order to choose the suitable image acquiring system and create comparison table. This system is an alternative way to reduce time and process in the image acquiring process in technical photography technique.

Image-based 3D face reconstruction has a huge application in the field of face analysis, such as face recognition, facial animation, and face editing. Recently, the popular methods based on 3dmm suffer from the ill-posed face pose16 and depth ambiguity issue. In order to address the issue, two multi-view geometric constraints are included in the reconstruction process. Note that before using these two constraints, a complete UV texture needs to be generated by texture fusion. Then, we can establish dense correspondences between different views leveraging a novel self-supervised pixel consistency constraint. We also use the facial landmark-based epi-polar constraint to constrain the pose between different views to obtain more accurate results. Extensive experiments demonstrate the superiority of the proposed method over other popular 3dmm methods with single-view input in accuracy and robustness, especially under large poses.

Super-resolution of single image aims to reconstruct its high-resolution version from a single low-resolution image. The existing reconstruction methods are inadequate in edge preservation and edge artifact suppression. In this paper, we propose a new SR method based on the sparsity along the directions of image gradients and the similarity of directional features. First, the Directionlet transform is used to extract the directional features of the image; then the directional total variation regular terms and the similarity weight calculation of the non-local mean are applied to the extracted directional features, resulting in that the detailed features of the image can be preserved effectively and the edge artifacts can be suppressed better. Finally, we use a framework of templates for first-order conic solvers with an energy function of minimal error to reconstruct the SR image. Experimental results show that our method that the superiority of our proposed method over the state-of-the-art algorithms.

Deep Learning (DL) techniques such as convolutional neural networks (CNNs) can achieve profoundly reliable feature extraction results when fitted with large and enough data sets and respective labels. Still, applying CNNs concurrently with insufficient labeled data can be problematic as this leads to overall overfitting. In addition, tropical cyclone intensity estimation has been greatly studied and several techniques have been devised although it remains an open challenge for it needs domain understanding during the extraction of feature vectors and significant pre-processing of diverse sets of parameters. Therefore, in this study, we introduce a novel method for tropical cyclone intensity estimation based on infrared images. The proposed method is trained directly on tropical cyclone infrared (TCIR) hourly satellite images. We performed the principal component analysis (PCA) transformation on these extracted feature vectors to reduce the data dimensionality. Ultimately, the transformed features with their corresponding wind speed (kts) features are analyzed using support vector regression (SVR) to ascertain their existing intensity. The experimental results show a strong linear relationship between the two variables. Thus, it indicates that our proposed approach performed remarkably well with a root-mean-square error of 9.65 knots (kts).

This paper presents a novel Res2Net multi-scale image super-resolution (SR) network in the curvelet transform domain based on deep learning, namely curvelet-Res2Net. Firstly, we integrate Res2Net block to adaptively increase the range of receptive fields for each network layer and exploit its potential multi-scale ability at a granular level from images. And then, a sequence of cascaded Res2Net blocks is formed into curvelet-Res2Net. In addition, the image SR problem is formulated as curvelet coefficient prediction to preserve richer structure details to overcome the drawbacks of oversmoothed outputs and poor texture details of most previous high resolution (HR) image predication methods in the spatial domain. Three standard image datasets are used qualitatively and quantitatively verify the performance of our method, and the results show its capability of achieving higher Peak Signal-to-Noise Ratios (PSNRs) and Structural SIMilarity (SSIM) while preserving much more useful information, comparing with other methods.

Deep learning has been widely used in super-resolution reconstruction tasks in recent years. Most of the work is based on external examples, these methods have made great progress by training mapping functions from low-resolution (LR) image patches to high-resolution (HR) image patches compared with traditional methods. There are also a few methods focus on a single image and use the internal examples to get high-resolution images. The method based on prior knowledge obtains a large number of nonlinear mapping functions through complex convolution kernels, and significantly improves the reconstruction performance of the super-resolution task. However, these external example- based methods require a large number of patch pairs to train network parameters. Besides, most of the LR images are down-sampled from the ground truth images, not all the LR images in the real world come from the HR images, these images may be disturbed by noise, blurring and other factors, some LR images do not even have a corresponding ground truth image. These shortcomings make the training of the methods based on prior knowledge very time-consuming, and the reconstruction performance of specific images uncertain. Zero-short SR(ZSSR) firstly combines deep learning and internal examples together, and get satisfactory HR images at the test time. However, compared with the methods based on prior knowledge, ZSSR only uses the single image itself as the training dataset, directly learning the mapping functions between LR image patches and HR image patches does not fully display the self-similarity within the single image. In this paper, we further combine the internal mapping with deep-learning, learning internal mapping from different scales to get HR images with more fine details.

A method for separation of occluded persons in thermal infrared images is proposed. Firstly, an infrared image is preprocessed by a suitable segmentation method, so that the binary image, which indicates the regions of occluded persons, is available. Then, via mediod-shifts of spacial coordinates, the proposed method classifies the pixels of each region into medoid regions without prior knowledge and therefore adapts well to complex occlusion patterns in applications of infrared human detection.

The chlorophyll-a concentration is one of the main indicators to evaluate water quality and measure the degree of eutrophication of the water body. It is of great significance to the environmental protection of rivers and lakes. At present, the commonly used empirical/semi-empirical remote sensing inversion models have low inversion accuracy in rivers with low chlorophyll-a concentration and narrow chlorophyll-a concentration ranges. At the same time, the semianalytical models require specific wavelengths, which makes it more suitable for hyperspectral remote sensing data. But it is difficult to obtain high-resolution hyperspectral satellite data. Taking the Danjiangkou-Fancheng reach of the Han River as the research area, this paper establishes a partial least squares regression model to estimate chlorophyll-a concentration using Sentinel-2 multispectral remote sensing images and synchronously measured chlorophyll-a concentration data. Considering the impact of Wangfuzhou Hydropower Station on the upstream and downstream, a segmented modeling was adopted and verified. The results show that the average relative errors of the verification samples in the Danjiangkou-Wangfuzhou reach and Wangfuzhou-Fancheng reach are 22.53% and 11.75%, respectively and the root mean square errors are 0.65ug/L and 0.17ug/L, respectively. The prediction accuracy is high. It indicates that the partial least squares method can be applied to the inversion of chlorophyll-a in rivers with low chlorophyll-a concentration and narrow chlorophyll-a ranges based on multispectral data.

The booming development of deep convolutional neural networks recently has made super-resolution researches achieve great progress. However, most of the existing researches only consider the super-resolution of several integer scale factors. In this paper, we propose a dual-scale convolutional neural network (DSCNN) to solve super-resolution problem of arbitrary scale factor. The magnifying module of the proposed DSCNN is designed with two chains. First, the large- scale chain learns the feature mappings from the image blocks of the low resolution (LR) to those of the high resolution (HR). The LR image blocks are the magnified image blocks with the desired size via bicubic interpolation. Second, the small-scale chain learns the feature mappings from the down-sampling image blocks to the magnified image blocks. Compared to the existing SR networks, DSCNN has two advantages: (1) it can handle the super-resolution images with arbitrary scale factors, and (2) it dynamically predicts the values rather than the weights of the interpolated pixels of HR images. The extensive experiments on widely used benchmark datasets show the superiority of the proposed DSCNN to the state-of-the-art SR methods in terms of both numerical results and visual quality.

Convolutional neural network is widely used in image fusion. However, the deep learning framework is only applied in some part of the fusion process in most existing methods. To generate a full end-to-end image fusion pipeline, a Yshaped Generator model based on Generative Adversarial Network for infrared and visible image fusion is proposed. The idea of this method is to establish an adversarial game between the generator and the discriminator. The generator consisting of two Pyramid networks and three convolutional layers works as an autoencoder to improve the characteristic information of the fused images. As for the discriminator, it adopts a network structure similar to the Visual Geometry Group (VGG) network. The loss function uses the ratio loss to control the trade-off among generation loss and reconstruction loss. Results on publicly available datasets demonstrate that our method can improve the quality of detail information and sharpen the edge of infrared targets.

Recently channel attention mechanism has been widely used to improve the performance of convolutional neural networks. However, most channel attention mechanisms applied to the backbone convolutional neural networks of the computer vision use the global pooling features of the output from each block to obtain the channel attention weights of corresponding channels, ignore the spatial information of the corresponding original features and the potential relationship between adjacent layers. For insufficient utilization of space information of origin features and inability to adaptively learn the potential association of all features in a block before the process of producing channel attention weights, we propose a new Cross-layer Channel Attention Mechanism（CCAM）, in which a matrix with spatial information is used to replace the global pooling operation, uses the input and output features of each block as the inputs, and outputs the channel attention weights of corresponding features simultaneously. Compared with other attention mechanisms, the CCAM have the following three advantages: first, it makes full use of the spatial information of each layer of features; second, it encourages feature reuse and fusion; third, it is better at discovering the potential relationship between the features of different layers in a block. Our simulation results have demonstrated that CCAM can effectively extract the attention weights of diffident layers, and achieve better performance on CIFAR- 10, CIFAR-100, ImageNet-1K, MS COCO detection, and VOC detection with small additional computational cost compared with the corresponding convolutional neural network.

With the development of wearable cameras a new environment has emerged, the egocentric perspective, and with it the computer vision task of detecting the hands and disambiguating them left from right. In order to address this challenge, we use an Attention Network with various egocentric hand properties to make the final classification. These hand features are inspired by the egocentric perspective and include the hand location in the image, the hand size, the fact there is at most only one object of each hand class and the probability of each hand to appear in the image. In addition, we use the YOLO object detector and its Tiny version to see their impact on the overall performance and speed, which is needed for wearable devices. Finally, we compare them with current object and hand detection approaches.

It’s well known that various extents of discontinuous artifacts often occur in reconstructed video. Massive in-loop coding algorithms have been presented to reduce artifacts. However, when bitrate is insufficient, in-loop coding tools alone fail to solve the problem properly. Preprocessing can be served as an effective solution to reduce compression distortion at low bitrate. In this paper, we propose a novel re-cursively adaptive perceptual non-local means (RAP-NLM) preprocessing algorithm based on just noticeable distortion (JND) model. By recursively employing both spatial and temporal non-local content perceptual characteristics, RAP-NLM filter could be adapted to reduce perceptual redundancies, which will help alleviate the artifacts. Experimental results show that our adaptive perceptual preprocessing algorithm can effectively improve the perceived quality of reconstruction video frames.

Comparative research was carried out on three methods of acquiring visual data on the shape of obstacles in wastelands and forest areas for automatic control of the stability of self-propelled machines. The tests were carried out for the following sensors: ultrasonic, 3D IFM O3M251 sensor, and Intel RealSense D455 sensor. The sensors were mounted on a common beam and positioned inline above the obstacles. Four cylindrical objects of various structures and sizes, placed on a driven belt conveyor, were used as recognition obstacles. Tests were carried out at two operating speeds. As a result of the research, obstacle heights were identified concerning relation to their movement under the sensors on the conveyor belt. The obtained results allowed for the assessment of the suitability of each of the methods in the context of their use as a substrate monitoring system for the stabilization systems of agricultural and forest self-propelled machines.

Generative neural network models have been used to create impressive synthetic images. However, artificial video synthesis is still hard, even for these models. The best videos that generative models can currently create are a few seconds long, distorted, and low resolution. We propose and implement a model to synthesize videos at 1024 × 1024 × 32 resolution that include human facial expressions by using static images generated from a Generative Adversarial Network trained on human facial images. To the best of our knowledge, this is the first work that generates realistic videos that are larger than 256 × 256 resolution from single starting images. Our model improves video synthesis in both quantitative and qualitative ways as compared to two state-of-the-art models: TGAN and MocoGAN. In a quantitative comparison, we achieve a best Average Content Distance (ACD) score of 0.167, as compared to 0.305 and 0.201 for TGAN and MocoGAN, respectively

Inferring the three-dimensional structure of an object or scene from multi-view images is a hard inverse problem. In this paper, we propose a depth map-based method to reconstruct a 3D model from multiple calibrated images. The proposed method uses global and local optimization in two stages to estimate highly accurate depth maps. The first stage employs Particle Swarm Optimization to explore the search-space extensively and estimate coarse depth maps from downscaled images. A cross-view depth map completion step ensures that good depth estimates are transferred to neighboring views. This helps in removing outliers and fill in missing depth values with reliable estimates. In the second stage, a local optimizer refines the coarse depths by processing each row/column of pixels iteratively to improve photo-consistency and smoothness among neighboring depths. This procedure utilizes distance maps to handle textured and homogeneous regions adaptively. Experimental results on the Middlebury multi-view stereo benchmark demonstrate the effectiveness of our method in producing accurate and complete 3D models with better spatial consistency and level of detail.

Robotic grasping in multi-object stacking scenes is important for autonomous robot manipulation. In this paper, we propose a 6-DoF (Degree of Freedom) grasping method for stacked rectangular objects from the single-view point clouds. We use the PointNet++ network and DBSCAN clustering algorithm to extract the target object from the whole scene. The 6-DoF pose of the gripper is obtained by our grasp pose estimation algorithm. To train the PointNet++ network, we build a small rectangular object segmentation dataset containing 800 real-world stacking scenes. The whole grasping system is lightweight, which takes about 518ms for a whole grasp planning process. Sufficient experiments show that our method gets 92% success rate and 95.5% completion rate, which satisfies the requirements of industrial applications.

Image colorization is a technology to transform gray-scale image into natural color image, which is always a challenging task in image processing. In this paper, we design a deep neural network model including multi-scale convolutional down-sampling and attention mechanism, which can effectively solve the problem of coloring gray remote sensing image. First, we input the grayscale remote sensing image into the feature extractor of our model for feature extraction and multi-scale down-sampling. Second, the feature map extracted by the feature extractor is input into the squeeze-andexcitation attention mechanism module to enhance the extraction of key information. Finally, the feature map output by the feature extractor and squeeze-and-excitation module is input into the feature reconstructor for feature reconstruction and color restoration, and then color remote sensing images are obtained. Unlike the conventional methods, our method is an end-to-end model, and without any human interaction in the coloring process. The experimental results show that the color remote sensing image generated by our method is superior to the existing methods in both subjective and objective metrics.

With an ever-increasing amount of image data, the manual labeling process has become the bottleneck in many machine learning applications. Plankton taxa labeling is especially a challenge due to its complex nature, and the manual labeling effort places a large burden on the domain experts. The Active Learning (AL) paradigm is a promising research direction adopted in the literature to minimize the manual labeling effort exerted by domain experts. Many approaches for AL have been proposed over the recent years to improve the labeling task by supporting the construction of large datasets suitable to train machine learning models while minimizing human involvement in the process. Our empirical study suggests that many modern active learning methods fail to incorporate both the samples that represent the statistical pattern of the data and the samples in which the machine learning model is not confident about. Inspired by these limitations, we propose an algorithm that combines these two types of sampling in order to capture the data distribution of the whole feature space, prevent redundant sampling from correlated uncertainty queries and finetune the inter-class decision boundary. Our experiments show that the proposed method outperforms each of the methods separately. Further, it also proves to be efficient on both the CIFAR dataset and the more complex Kaggle plankton dataset.

Application of more and more advanced information technologies in agriculture comprises broader and broader range of production processes, planning processes, monitoring and marketing processes. The applied information technologies are used in production technology of animals and crop production. Within the recent decades one can observe a dynamic development of research on artificial intelligence, and at the same time the development of research within the range of advisory systems (expertise), as well the use of methods of computer image analysis in the process of quality assessment/evaluation of vegetables and fruits. One of the areas of using computer image analysis is supporting decision making processes within the range of quality evaluation of agri-food products. The aim of the project was to use image analysis and artificial neural networks to classify quality of convective dried carrot.

Makeup, derived from human`s pursuit of beauty, is widely accepted by the public. Despite its popularization, there are little effect been made to tackle the makeup face verification challenges. Aiming to promote existing verification system to accept or reject the claimed identity of a person with makeup in an image, a makeup robust face verification framework is proposed based upon a generative adversarial network. The proposal synthesizes non-makeup face images from makeup images for further verification. Specifically, a patchwise contrastive loss is introduced in the generative model to narrow the distance between makeup and non-makeup images. The challenge in the state-of-the-art is the employment of a pre-specified and hand-designed loss function to measure the performance, which is not the case in the proposal. Experimental results demonstrate that the proposal generates non-makeup faces with few artifacts and achieves 96.3% accuracy on Dataset1 in face verification, which is at least 0.8% better than some well discussed models.

Quantum image processing and its various applications developed in the last few years. These applications utilize different quantum algorithms for higher accuracy and reduced time complexity compared to classical computing. This paper reviews some applications of quantum image processing like edge extraction, Security, and denoising of images. Applications like feature extraction, machine vision, and feature detection use edge extraction. In this paper, we observe the field of Security by dividing it into two sub-areas, i.e., cryptography and information hiding, used for authentication purposes. For removal of noise signals from input images, the method of denoising is used.

With the improvement of modernization and intelligentization of dam construction sites, how to quickly identify the particle size distribution of mixed aggregates at the construction site of cemented particle dams is an urgent problem to be solved. This paper took the aggregate particle size and gradation distribution of the cemented particle dam as the research object, and developed a mixed aggregate gradation sampling detection device integrated by LED light source, industrial CCD camera and software platform, using digital image processing technology and the equivalent volume algorithm realized the digital, intelligent and automated non-contact rapid detection of aggregate gradation at the construction site. The test showed that the cumulative pass rate experimental error was below 4%, the correlation between the detected sand and gravel quality and the actual quality reached 0.96, and the gradation curve trend of the test results was consistent with the actual measured trend. It not only proved the reliability and effectiveness of the mixed aggregate gradation sampling detection device developed in this paper to quickly identify the particle size distribution of mixed aggregates in the construction environment, but also proved the accuracy and rapidity of the gradation detection method in this paper. It was of great significance to the construction of cemented particle dams in terms of economy, environmental protection and safety.

In view of the fusion of SAR image and RGB image, different methods based on wavelet transform (WT) were studied. Based on WT, SAR images are fused with different channel of RGB images respectively. In order to suppress the speckle in SAR images, Speckle Reducing Anisotropic Diffusion (SRAD) and nonlocal mean filter (NLM) were introduced respectively. The image fusion effect was comprehensively measured by using several image quality indexes. The results show that WT and NLM are more helpful to improve the quality of image fusion.

The aim of the research presented in this work was to develop a model of Artificial Neural Networks (ANN) with the use of computer image analysis for the qualitative classification of deep-frozen raw material - breaded pollock cutlets (Backfisch). Shape and color discriminants were selected, by using a computer program, it was possible to obtain numerical data and build a learning set from them. This work is an example of using one of the methods of artificial intelligence in the food industry. The designed network was characterized by a very high ability for classification, its training was done by the technique of the so-called " with a teacher". Such actions are motivated by the requirements of consumers who are becoming more and more attentive to the products they consume and expect calorically balanced and very high quality products.

A new method to measure air quality using image blur metric was proposed. The classification of blur degree based on local image blocks helps to achieve autonomy selection of the visual attention region. Then the relationship between image blur degree and the concentration of air particulate matters was analyzed. The results show that the correlation between the image blur degree and the PM2.5 concentration is the best, and the correlation coefficient reached 0.76. Therefore the image blur degree evaluation can be used as a measure of air quality.

Anomaly detection aims to find different patterns from those seen previously. It is usually regarded as a oneclassification problem where abnormal classes are often scarce or not well defined, while the target class (training objects) are sufficient. Recently, several methods have achieved excellent performance through an auxiliary multi-class task(such as rotation predict) used in self-supervised learning. However, these classification-based approaches which adopt the crossentropy loss have inherent defects in anomaly detection. Specifically, the relative measure of cross-entropy may result that a normal sample suffered from a low score would be misclassified as an abnormality. In order to solve this problem, we propose an Absolute Measurement Anomaly Detection (AMAD), to constrain the distribution of activations of each input in the classification network. In details, this technique encourages the output of the ground truth class to be higher, vice versa for unrelated classes. Furtherly, different from the previous evaluation methods that counting the log-softmax activation of the model as normality score, we ignore the log-softmax since the score would be affected heavily provided that more misclassification occurs. We present experiments both in image datasets(CIFAR-10, Fashion-MNIST) and tabular datasets(KDDCUP et al.), which show that our technique achieves better performance in terms of AUROC and F1 score when compared with previous similar methods.

Estimating a scene's depth from image pairs is a problem for many computer vision applications such as autonomous vehicles and 3D object reconstruction. However, traditional methods propose dissimilarity functions that do not solve ambiguity problems. We present a combined siamese convolutional neural network (CSCNN) approach to calculate the costs of disparities in stereo images using patches with different sizes. This approach can learn the patches context, improving the accuracy of the estimated costs. We apply the semi-global matching method and the median filter to increase further the robustness of matching costs. We trained and evaluated our approach using the Middlebury database, and, through the bad pixel evaluation, we demonstrate that our approach achieved an accuracy improvement of approximately 22%, compared to the results obtained by single-window CNN-based approaches.

This paper presents a face control system for 3D avatar with webcam using the Facemesh API for face tracking and Jeeliz-transfer API for eye tracking. 3D avatar face animation is developed as a responsive web application. It starts with face detection and tracking through the webcam. Face coordinate data is normalized to a vertical face view where the distance between the eyes and the level of the eyes are the same for each video frame. Then new face coordinates are calculated in both 2D and 3D to study the change of specific coordinates such as mouth shape and face shape. In addition, the coordinates of face structures in the video platform have also been added to the 3D model platform. Specific coordinates are studied to analyze distance changes to be applied in 3D avatar manipulation. The 3D models are designed and created in multiple blend-shapes or basic character facial features. However, due to the limitations of web browser-based 3D morphing, which Three.js is used for morph target displays, it allows only eight combination shapes to be displayed at the same time. Therefore, the required blending geometry must be pre-assembled. Blend-shape factors are based on an analysis of the coordinates of each moving face in order to eliminate the limitations. Our 3D facial animations with a webcam generate results of high quality, real-time and online simulation. Therefore, our work is a fundamental technology that can be applied to animate other 3D characters in blend-shape format.

In this study, Hyperspectral Imaging System (400-1,000nm) was used to discriminate the maturity of peanut cluster. Hyperspectral data of 480 peanut samples in 4 different mature stages were collected by Hyperspectral Imaging (HSI) system, respectively. Principal component analysis (PCA) was conducted on the hyperspectral images. Support vector machine (SVM)was used established discriminant model using the spectra basing on feature wavelength (extract the 4 feature wavelengths were 415, 575, 727, and 930 nm). The results of the discrimination model depending on PCA indicated that the different mature was accurately identified with a calibration accuracy of 94.5% and a prediction accuracy of 89.1%; the results of the discrimination model based on feature wavelength was accurately detected with a calibration accuracy of 93.5% and a prediction accuracy of 89.5%. The overall results indicated that an HSI technique was constructed from spectral and image information both aspects of information, which can be combined with multivariate analysis could be used as an effective tool for non-destructive and rapid variety discrimination and visualization of different mature stages peanut cluster in one plant. The proposed method provides great potential for developing a multi-spectral imaging system for practical application nondestructive testing of peanut cluster maturity. This study contributes theoretical guiding significance and can be applied for the further identification of peanut maturity in the field and industry of the peanut production chain.

Managing pain is a fundamental care need, of which inadequate management may have deleterious short- and long-term consequences to patients. In adult patients, pain is commonly assessed by an individual reporting on the numerical rating scale. However, self-reporting of pain may not be possible for patients with cognitive dysfunction, or those who are burdened by medical conditions e.g., stroke, degenerative diseases, damaged vocal cords, critically ill patients etc. Similarly, Pre-verbal children, and those with profound cognitive impairment are unable to self-report and describe their pain level, making pain assessment more challenging. Difficulty in self reporting in pain assessment is a significant clinical problem that affects clinical decision. Therefore, automatic pain detection is important for patient care. In this paper, we propose a Spatial-temporal Deep Neural Network (DNN) to detect the pain level of a patient from the facial landmarks. The proposed system firstly detect face from a frame, and then extract facial landmarks from face detected, after that a series of frames containing facial landmarks are fed into the 3D convolutional networks for high level spatial and temporal feature extraction, and finally classify into 3 categories: no pain, mild pain and serve pain. The proposed system achieved 97% accuracy after being trained and tested using UNBC-McMaster Pain Archive Database.

This paper focuses on offline writer-dependent signature verification using a Siamese convolutional neural network. A Siamese network is comprised of equal-weighted twin networks that can be trained to learn a feature space in which similar observations are juxtaposed. To reduce the impact of the high intra-variability of the signature and ensure that the Siamese network is able to learn more effectively, we propose a method of selecting a Reference (REF). Using the proposed reference selection, the accuracy can be increased by 6.4%. By utilizing the GPDS-160 signature data-set, the designed system is able to achieve an accuracy of 94.5%, which is a better result than that achieved by current state-ofthe-art writer-dependent techniques.

Recently the demand for fruit and vegetable juice powders has increased significantly as there are numerous benefits of using these products in various forms of food. Therefore, it is important to optimise spray drying techniques and find how processing factors influence the quality of powders. For this reason, researchers seek modern methods to aid the assessment of quality of food powders. In this study classes of raspberry powders were distinguished on the basis of selected physical parameters such as: colour expressed in the CIE L* a* b* system, moisture content, and water activity. The classification accuracy of the neural models developed in this study was over 96%.

Semantic segmentation is a task that traditionally requires a large dataset of pixel-level ground truth labels, which is time-consuming and expensive to obtain. Recent advancements in the weakly-supervised setting show that reasonable performance can be obtained by using only image-level labels. Classification is often used as a proxy task to train a deep neural network from which attention maps are extracted. However, the classification task needs only the minimum evidence to make predictions, hence it focuses on the most discriminative object regions. To overcome this problem, we propose a novel formulation of adversarial erasing of the attention maps. In contrast to previous adversarial erasing methods, we optimize two networks with opposing loss functions, which eliminates the requirement of certain suboptimal strategies; for instance, having multiple training steps that complicate the training process or a weight sharing policy between networks operating on different distributions that might be suboptimal for performance. The proposed solution does not require saliency masks, instead it uses a regularization loss to prevent the attention maps from spreading to less discriminative object regions. Our experiments on the Pascal VOC dataset demonstrate that our adversarial approach increases segmentation performance by 2.1 mIoU compared to our baseline and by 1.0 mIoU compared to previous adversarial erasing approaches.

To solve the problem of specific emitter identification (SEI) under the Rice channel, an SEI method based on full bispectrum and deep residual shrinkage networks (DRSN) is proposed. The method uses phase noise to model individual differences and utilizes non-parametric methods to extract the full bispectrum of the signal. Then the full bispectrum is used as the input of convolutional neural networks (CNN) and the model learns signal features and completes the classification task through multi-layer cascaded DRSN subsequently. The simulation experiment models 3 different radiation sources, and the modulation mode is set to BPSK, MSK, OQPSK. Experimental results show that the method we proposed can effectively identify different radiation sources, and also has a high recognition performance under low SNRs. Compared with other methods that use reduced-dimensional bispectrum features as input, the recognition accuracy is effectively improved.

Aiming at the problems of high cost, low efficiency, lack of systematic management and intelligent monitoring, a set of remote water quality monitoring system based on Lora technology and 4G Network is designed. The system uses solar battery as power supply, and F8L10T_LoRa as acquisition terminal module. The data of DO (dissolved oxygen), PH and temperature collected by the sensors are uploaded to F8926L_Router gateway. The gateway uses 4G Network to communicate with upper computer software. And the Kingview software is adopted to display and store the data in real time. The results show that the system can accurately obtain the data of each pond in real time, and the packet loss rate is less than 2% in the range of 5 km during the three-month test.

Many video encoders use DCT transform coding to compress the encoded video. For hardware implementation, DCT will be approximately an integer matrix, which may cause some deviations in this process, and these deviations will accumulate and become obvious in the larger code unit. Our method is to construct all DCT-related discrete orthogonal transforms in the required size (corresponding to the coding unit supported by H.266/VVC). By using a novel discrete orthogonal matrix generation method with determined DCT-II roots, and scaling and rounding a regular DCT that depends on the quantization parameter, instead of integer approximation. We can obtain an accurate integer DCT matrix. Experimental results show that this method can not only improve the video quality and also require fewer bit rates.

The compact intensity-modulated frequency modulated continuous wave (FMCW) laser range sensor has a low signal-tonoise ratio(SNR) because the transmitting power is limited by heat dissipation conditions, and the detection performance is reduced in degraded visual environments. To increase the transmitting power and keep the heat low, the SNR enhancement technique using synchronized dual-laser sources is proposed. This paper established a mathematical model of the echo signal of the dual-laser sources, made a comparative analysis of the single and dual-channel laser beat signal. Aimed at the problem of unsynchronization of the two modulated signals caused by the drive circuit, a phase synchronization algorithm based on sampling feedback was proposed and verified by simulation. Finally, a synchronized dual-channel laser ranging system was built in the laboratory for experiments. Compared with a single-channel laser, the synchronized dual-channel laser transmitting power is doubled, and the echo signal SNR can be increased by 2.2dB. The experimental results indicate that the proposed SNR enhancement technique can effectively solve the problem of low SNR in compact intensity-modulated FMCW laser range sensor.

The performance of Synthetic Aperture Radar (SAR) ship detector has been significantly improved with the development of convolutional neural network. However, the issue of effective detection of inshore ships is still a challenging problem. In this paper, we propose a novel one-stage SAR ship detector, called Semantic Attention-Based Network (SANet), which can largely improve the accuracy of ship detection in the inshore scenario without compromising the speed. Specifically, we introduce a semantic attention mechanism, which will highlight the features from the ships area and enhance the detector's classification ability. We train the proposed Semantic Attention Module with focal loss, and assign labels for the attention maps by center sampling. Combined with our anchor assign strategy, our SANet achieves state-of-the-art results on the open SAR Ship Detection Dataset (SSDD).

Soil moisture is an important parameter in the surface process, and it is indispensable in the field of crop growth and drought monitoring. SAR can penetrate clouds and fog to achieve high-resolution observations, so it has great advantages in remote sensing estimation of soil moisture. In this paper, Sentinel-1A radar data and MODIS were used to explore the applicability of soil moisture retrieval in a large area based on the support vector regression (SVR) method. By analyzing the characteristics closely related to soil moisture, the input of the algorithm was determined including VV, VH polarization backscattering coefficient, radar local incident angle (LIA), digital elevation model (DEM), slope (SLP), and normalized vegetation index (NDVI). Then the accuracy of the SVR model constructed with different feature combinations was discussed, and the best performing model was selected to estimate soil moisture in northern and central Anhui province. The results showed that the model with the combination of VV, LIA, NDVI, DEM, and SLP input had the highest accuracy with R² of 0.9413 and the root mean square error (RMSE) of 0.0085 cm³·cm^-3 , in which terrain factor had a greater impact. Finally, the best model was used to achieve a wide range of soil moisture retrieval, and test samples were used to verify the estimation accuracy with R² of 0.6444 and RMSE of 0.036 cm³·cm^-3 . Moreover, the temporal and spatial distribution of the retrieval result is reasonable, which can characterize the distribution difference of a large study area.

The evaluation and optimization of reservoir operation schemes belong to a multi-objective, multi-level and multi-attribute decision-making problem. The reservoir multi-objective dispatching model generates many Pareto feasible solutions, and decision-makers often make decisions difficult. The traditional scheme selection method has the problems that the index weight is greatly affected by subjectivity, the single weight determination method is one-sided, the model calculation is complicated, and the characteristics of the evaluation index cannot be fully extracted. Therefore, this paper proposes a new method for reservoir operation plan optimization based on fuzzy optimization and convolutional neural network. First, establish the evaluation index system of the reservoir operation plan based on the fuzzy optimization theory, select the analytic hierarchy process to determine the subjective weight of the index, the entropy weight method to determine the objective weight, use the game theory to couple the subjective and objective weights, and calculate the comprehensive evaluation value of the plan through fuzzy comprehensive evaluation. Secondly, the evaluation index and comprehensive evaluation value are used as the input and output of the convolutional neural network to establish the optimal model of the reservoir operation plan. The results of the case analysis show that the research method has high accuracy and reliability, and can provide a scientific basis for reservoir operation decision-making.