Image inpaintnig in textile manufacturing is a new emerging research topic in preprocessing for jacquard CAD systems. One of the most important aspects of a jacquard CAD system is the simulation of the appearance of a jacquard texture during inference. Jacquard image inpainting has become an indispensable process for the Jacquard CAD system. Jacquard image reconstruction is designed to restore a damaged image with missing information, so it is necessary to determine which parts of the image need to be repaired. Thus, this task includes two processing stages: the detection of defects and their recovery. This article presents a two-stage approach that combines new and traditional algorithms for detecting defects and repairing damaged areas. The first stage is a defect detection method based on a convolutional autoencoder (U-Net). The second stage is image inpainting based on exemplar-based concepts and the anisotropic gradient. Our system quantitatively outperforms state-of-the-art methods regarding reconstruction accuracy in the benchmark.
A system for determination the distance from the robot to the scene is useful for object tracking, and 3-D reconstruction may be desired for many manufacturing and robotic tasks. While the robot is processing materials, such as welding parts, milling, drilling, fragments of materials fall on the camera installed on the robot, introducing unnecessary information when building a depth map, as well as the emergence of new lost areas, which leads to incorrect determination of the size of objects. There is a problem comprising a decrease in the accuracy of planning the movement trajectory caused by incorrect sections on the depth map because of incorrect distance determination of objects. We present a two-stage approach combining defect detection and depth reconstruction algorithms. The first step is image defects detection based on convolutional auto-encoder (U-Net) and deep feature fusion network (DFFN-Net). The second step is a depth map reconstruction with the exemplar-based and the anisotropic gradient concepts. The proposed modified block fusion algorithm uses a local image descriptor obtained by an automatic encoder for image reconstruction, which extracts image features and depth maps using a decoding network. Our technique outperforms the state-of-the-art methods quantitatively in reconstruction accuracy on RGB-D benchmark for evaluating manufacturing vision systems.
The article an approach to improve the accuracy of restoring the boundaries of objects obtained to create 3D structures by The paper proposes a data processing algorithm that allows performing primary processing operations in order to identify the main parameters and fusion them into a single image. To form a complex image, it is possible to first enter the parameters selected by the operator, which corrects the mixing ratio. As a noise reduction algorithm for different ranges, the multi-criteria filtering method is used, which is based on minimizing the sum of the squared deviations of the input signal and the generated estimate, as well as the sum of the squared differences of the obtained estimates. Using the adjustment factor allows you to set the degree of influence of the criterion on the resulting processing. Using this method also allows you to detect the boundaries of objects. The search for the border is based on the analysis of frequency components and the search for sharp changes in color gradation. The possibility of applying this approach for various types of data is shown on the example of processing parallel streams. For the primary construction of areas of significance, an algorithm for changing the range of clusters and an object complexity analyzer are used. The analyzer is built on the basis of calculating the weighted value of the number of color gradient transitions per unit area. To visually improve the quality of the data, a color space conversion algorithm based on alpha mixed is used. As test data used to evaluate the effectiveness, pairs of test images are used, obtained by sensors fixed at resolution of 1024x768 (8 bit, color image) and far-IR spectrum 320x240 (8 bit, color image). Images of simple shapes are used as analyzed objects.
This article presents a two-stage approach, combining novel and traditional algorithms, to image segmentation and defect detection. The first stage is a new method for segmenting fabric images is based on Hamiltonian quaternions and the associative algebra and the active contour model with anisotropic gradient. To solve the problem of loss of important information about color, saturation, and other important information associated color, we use the quaternion framework to represent a color image to consider all three channels simultaneously when segmenting the RGB image. In the second stage, our crack and damage detection method are based on a convolutional autoencoder (U-Net) and deep feature fusion network (DFFN-Net). This solution allows localizing defects with higher accuracy compared to traditional methods of machine learning and modern methods of deep learning. All experiments were carried out using a public database with examples of damage to the TILDA fabric dataset.
The article presumes a data processing algorithm that improves the accuracy of recognition of radio-electronic components in devices for automated installation. The paper proposes the use of a multicriteria filtering method that allows you to automatically change the smoothing coefficient. Varying the coefficient allows both reducing the noise component and preserving the boundaries of the radio elements without blurring. In order to enhance the contours of objects, a data simplification method is applied using the technique of reducing the range of clusters of color gradient histograms while preserving the shapes of objects. At the stage of detecting the boundaries of the elements and forming the structure of the elements of the radio component base, a modified one-dimensional two-criteria method is used. The combined analytical approach allows detection of the boundaries of radioelements and increases the productivity of the process. As test data used to evaluate the effectiveness, pairs of test images obtained by sensors fixed at various magnifications with a resolution of 1024x768 (8 bit, color image, visible range) are used. Images of simple shapes are used as analyzed objects
The article proposes an approach to improve the accuracy of restoring the boundaries of objects obtained to create 3D structures by analyzing data obtained by a machine vision system. At the first stage, the operation of reducing the number of color gradients is performed, the technique allows you to combine similar values into common enlarged structures. This operation allows you to simplify the analyzed objects, since small details are not important. In parallel with the first operation of denoising is performed. The paper proposes the application of the multicriteria processing method with the possibility of smoothing locally stationary sections and preserving the boundaries of objects. As an algorithm for strengthening the boundaries of objects, a modification of the combined multi-criteria method is used, which makes it possible to reduce the effect of salt/pepper noise and impulse failures, as well as to strengthen the detected boundaries of objects. The resulting images with enhanced boundaries are fed to the input of the block for constructing three-dimensional objects. The data obtained by both a stereo pair and a camera based on 3D construction using structured light were used in the work. On a set of synthetic data simulating the work in real conditions, the increase in the efficiency of the system using the proposed approach is shown. Based on field data under conditions of interfering factors in the form of dust/fog, the applicability of the proposed approach for solving problems of increasing the accuracy of restoring the boundaries of objects obtained to create three-dimensional structures is shown. Images of simple shapes are used as analyzed objects.
Cracks (craquelure) and paint losses are the main types of deterioration of master paintings as they are ageing. We explore the potential of deep-learning-based methods for virtual restoration of paintings focusing on crack detection and their digital inpainting. For the crack detection stage, we develop a model that combines the benefits of multimodal convolutional (MCN) and autoencoder neural networks based on U-Net. The proposed model, dubbed U-Net multimodal convolutional, proves to outperform both MCN and U-Net architectures as well as the benchmark machine learning models for multimodal crack detection, both visually and in terms of objective performance measures. The second stage in our virtual restoration framework, the digital crack inpainting, is an adaptive adversarial network. The obtained virtual restoration results show clear improvement in comparison with the reference methods in this domain. Also, we propose an original way of training an adversarial neural network, which allows us to apply it more successfully in practice. A series of experiments shows encouraging results compared to the current state-of-the-art and confirms the huge potential of deep learning in crack detection and virtual restoration of master paintings.
Over time, crack pattern (craquelure) inevitably develops in paintings as a sign of their ageing, sometimes accompanied by larger losses of paint (lacunas). In restoration treatments, cracks are typically not filled in, and virtual restoration is often the only option to “reverse” the ageing of paintings, simulating their original appearance. Moreover, virtual restoration can serve as an important supporting step in decision making during the physical restoration. In this research, we investigate the possibility of applying deep learning-based methods for virtual restoration. In particular, our crack detection method is based on a convolutional autoencoder (U-Net), and we employ a generative adversarial neural network (GAN) to virtually inpaint the detected cracks. We propose an original way of training the GAN model for painting restoration, which improves its practical performance. A series of experiments shows encouraging results in comparison with known methods, and indicates huge potential of deep learning for virtual painting restoration.
As a popular topic in automation, fabric defect detection is a necessary and essential step of quality control in the textile manufacturing industry. The main challenge for automatically detecting fabric damage, in most cases, is the complex structure of the textile. This article presents a two-stage approach, combining novel and traditional algorithms to enhance image enhancement and defect detection. The first stage is a new combined local and global transform domain-based image enhancement algorithm using block-based alpha-rooting. In the second stage, we construct a neural network based on the modern architecture to detect fabric damage accurately. This solution allows localizing defects with higher accuracy than traditional methods of machine learning and modern methods of deep learning. All experiments were carried out using a public database with examples of damage to the TILDA fabric dataset.
Currently, modern achievements in the field of deep learning are increasingly being applied in practice. One of the practical uses of deep learning is to detect cracks on the surface of the roadway. The destruction of the roadway is the result of various factors: for example, the use of low-quality material, non-compliance with the standards of laying asphalt, external physical impact, etc. Detection of these damages in automatic mode with high speed and accuracy is an important and complex task. An effective solution to this problem can reduce the time of services that carry out the detection of damage and also increase the safety of road users. The main challenge for automatically detecting such damage, in most cases, is the complex structure of the roadway. To accurately detect this damage, we use U-Net. After that we improve the binary map with localized cracks from the U-Net neural network, using the morphological filtering. This solution allows localizing cracks with higher accuracy in comparison with traditional methods crack detection, as well as modern methods of deep learning. All experiments were performed using the publicly available CRACK500 dataset with examples of cracks and their binary maps.
Museums all over the world store a large variety of digitized paintings and other works of art with significant historical value. Over time, these works of art deteriorate, making them lose their original splendour. For paintings, cracks and paint losses are the most prominent types of deterioration, mainly caused by environmental factors, such as fluctuations in temperature or humidity, improper storage conditions and even physical impacts. We propose a neural network architecture for the detection of crack patterns in paintings, using visual acquisitions from different modalities. The proposed architecture is composed of two neural network streams, one is a fully connected neural network while the other consists of a multiscale convolutional neural network. The convolutional neural network plays a leading role in the crack classification task, while the fully connected neural network plays an auxiliary role. To reduce the overall computational complexity of the proposed method, we use morphological filtering as a pre-processing step to safely exclude areas of the image that do not contain cracks and do not need further processing. We validate the proposed method on a multimodal visual dataset from the Ghent Altarpiece, a world famous polyptych by the Van Eyck brothers. The results show an encouraging performance of the proposed approach compared to traditional machine learning methods and the state-of-the-art Bayesian Conditional Tensor Factorization (BCTF) method for crack detection.
This paper proposed a patch-based inpainting algorithm for depth map reconstruction using a stereo pair image. The proposed approach is based on a geometric model for patch synthesis. The lost pixels recovered by copying pixel values from the source based on a similarity criterion. We used a trained neural network to choose “best similar” patch. Experimental results show that the proposed method provides better results than the state-of-the-art methods in both subjective and objective measurements for depth map reconstruction.
Specular reflections are undesirable phenomena that can impair overall perception and subsequent image analysis. In this paper, we propose a modern solution to this problem, based on the latest achievements in this field. The proposed method includes three main steps: image enhancement, detection of specular reflections, and reconstruction of damaged areas. To enhance and equalize the brightness characteristics of the image, we use the alpha-rooting method with an adaptive choice of the optimal parameter-alpha. To detect specular reflections, we apply morphological filtering in the HSV color space. At the final stage, there is a reconstruction of damaged areas using adversarial neural networks. This combination makes it possible to quickly and effectively detect and remove specular reflections, which is confirmed by a series of experiments given by the experimental section of this work.
In many cases the rain and snow on an image significantly degrade the effectiveness of any computer vision algorithm, such as object recognition, tracking, retrieving and so on. The automated detection and removing such degradations in a color image is still a challenging task. This paper presents a new rain and snow removal method using low- and highfrequency parts of a single image. For this purpose, we use a color image multi-guided filter and anisotropic gradient in Hamiltonian quaternions. The quaternion framework is used to represent a color image to take into account all three channels simultaneously when inpainting the RGB image. Our results show that it has good performance in rain removal and snow removal.
Quality control of welded joints is an important step before commissioning of various types of metal structures. The main obstacles to the commissioning of such facilities are the areas where the welded joint deviates from acceptable defective standards. The defects of welded joints include non-welded, foreign inclusions, cracks, pores, etc. The article describes an approach to the detection of the main types of defects of welded joints using a combination of convolutional neural networks and support vector machine methods. Convolutional neural networks are used for primary classification. The support vector machine is used to accurately define defect boundaries. As a preprocessing in our work, we use the methods of morphological filtration. A series of experiments confirms the high efficiency of the proposed method in comparison with pure CNN method for detecting defects.
This paper presents a patch-based inpainting algorithm for image block recovery in block-based coding image transmission. The algorithm is based on a geometric model for patch synthesis. The lost pixels are recovered by copying pixel values from the source using a similarity criterion. We used a trained neural network to choose the “best similar” patch. Experimental results show that the proposed method outperforms widely used state-of-the-art methods in both subjective and objective measurements of image block recovery.
Due to an impressive progress in digital imaging reached over past decades, the importance of analog video systems as primary instruments of receiving, transmitting and storing data has been greatly reduced. However, there still exist large amount of data stored in analog formats on media affected by aging. In this paper, a new three-stage method for detecting and restoring blotches on a video sequence has been developed. The new method including the motion compensation, LBP calculations and data classification using the neural network is shown to have higher efficiency than commonly used ROD, SROD and SDI methods.
Digital video forgery or manipulation is a modification of the digital video for fabrication, which includes frame sequence manipulations such as deleting, insertion and swapping. In this paper, we focus on the detection problem of deleted frames in videos. Frame dropping is a type of video manipulation where consecutive frames are deleted to skip content from the original video. The automatic detection of deleted frames is a challenging task in digital video forensics. This paper describes an approach using spatial-temporal analysis based on the convolution with a bank of 3D Gabor filters. Also, we use the 3D Convolutional Neural Network for frame drop detection for preprocessed frames. Experimental results demonstrate the effectiveness of the proposed approach on a test video database.
Some of old photographs are damaged due to improper archiving (e.g. affected by direct sunlight, humidity, insects, etc.) or have a physical damage resulting on appearance of cracks, scratches on photographs, non-necessary signs, spots, dust, and so on. This paper focuses on detection and removal of cracks from digital images. The proposed method consists of the following steps: pre-processing, crack detection and image reconstruction. A pre-processing step is used to suppress a noise and small defects in images. For a crack identification we use modified local binary patterns to form a feature vectors, and a non-linear SVM for a crack recognition. The combined inpainting method using structure and texture restoration is applied at the image reconstruction step. Image inpainting is the process of restoring the lost or damaged regions or modifying the image contents imperceptibly. This technique detects and removes the horizontal, vertical, diagonal cracks and other defects on complex scenes of image. We implemented proposed method on some mobile platforms for automatic image enhancement. Presented examples demonstrate the effectiveness of the algorithm in cracks detection and removal.
This paper focuses on novel image reconstruction method based on modified exemplar-based technique. The basic idea
is to find an example (patch) from an image using local binary patterns, and replacing non-existed (‘lost’) data with it.
We propose to use multiple criteria for a patch similarity search since often in practice existed exemplar-based methods
produce unsatisfactory results. The criteria for searching the best matching uses several terms, including Euclidean
metric for pixel brightness and Chi-squared histogram matching distance for local binary patterns. A combined use of
textural geometric characteristics together with color information allows to get more informative description of the
patches. Texture synthesis method proposed by Efros and Freeman for patch restoration is utilized in the proposed
method. It allows optimizing an overlap region between patches using minimum error boundary cut. Several examples
considered in this paper show the effectiveness of the proposed approach for large objects removal as well as recovery of
small regions on several test images.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.