This PDF file contains the front matter associated with SPIE Proceedings Volume 11605 including the Title Page, Copyright information, and Table of Contents.

Autonomous driving considers issues related to a car driving in different real world situations. This work addresses a parking task and describes a new LIDAR-based parking spot search algorithm. The proposed approach was successfully validated in virtual experiments within the Gazebo simulator in a parking area with a perpendicular parking setup. HDBScan, OPTICS, and Gaussian Mixture clustering methods were compared for LIDAR data clustering in the parking spot search task, and the HDBScan clustering demonstrated best prediction and performance results.

This paper presents a novel approach for aerial drone autonomous navigation along predetermined paths using only visual input form an onboard camera and without reliance on a Global Positioning System (GPS). It is based on using a deep Convolutional Neural Network (CNN) combined with a regressor to output the drone steering commands. Furthermore, multiple auxiliary navigation paths that form a ‘navigation envelope’ are used for data augmentation to make the system adaptable to real-life deployment scenarios. The approach is suitable for automating drone navigation in applications that exhibit regular trips or visits to same locations such as environmental and desertification monitoring, parcel/aid delivery and drone-based wireless internet delivery. In this case, the proposed algorithm replaces human operators, enhances accuracy of GPS-based map navigation, alleviates problems related to GPS-spoofing and enables navigation in GPS-denied environments. Our system is tested in two scenarios using the Unreal Engine-based AirSim [32] plugin for drone simulation with promising results of average cross track distance less than 1.4 meters and mean waypoints minimum distance of less than 1 meter.

Through the ages, in all nations, at all times, people spend a lot of their time on discussing new and important issues either on meetings or in conferences. With the evolution and the abundance of Automatic Speech Recognition (ASR) frameworks, automatic transcripts and even automatic meeting summarization are getting more and more interesting. Recently, automatic summarization faces deeper progresses on speech summarization. Neural models had been introduced to tackle with many difficulties of abstractive summarization. Our contribution in this paper focuses on these weaknesses of neural abstractive meeting summarization and suggests an encoder-decoder model based on an attentional algorithm on the decoding sequence. We proposed a deep encoder-decoder model based on attention mechanism (DEDA) for ASR transcripts. Experiments on the AMI Dataset demonstrates that our proposed method ensured competitive results with the state of the art even on extractive or abstractive models. The experimental analyses also put the stress on the performance of the summarized utterances as well as the reduction of the occurrence repetition in summaries.

Semi-supervised learning techniques are gaining importance in the scenario of constantly growing data collections. CBIR systems must be able to autonomously analyze the patterns available, to fully exploit unlabeled data with the final objective of identifying an optimal representation space where data belonging to the same semantic class are close to each other. In this work we propose to adopt relevance feedback as a mean of collecting information about the semantic classes perceived by the user and to exploit this information for a long-term learning process where a more effective feature space can be obtained by a proper metric learning technique and class labels can be automatically assigned to unlabeled patterns. The process can iterate as new data become available thus providing a tool for successfully managing new incoming data. The experimental results will confirm the advantages of the proposed learning approach.

At present, most unsupervised abnormal behavior detection method only relies on powerful behavior detection classifiers, does not make full use of prior knowledge. This method often has the problem of a huge amount of calculation and affecting the detection speed. In view of the above problems, this paper proposes a weak anomalyreinforced autoencoder for unsupervised anomaly detection method, using U-Net to reconstruct video frames and generative adversarial network to learn the correlation between image entropy and abnormal behavior. Comprehensive experiments on the avenue data set and UCSD data sets verify the effectiveness of our method to detect abnormal events.

The implementations of the convolution operation in neural networks are usually based on convolution-to-GeMM (General Matrix Multiplication) transformation. However, this transformation requires a big intermediate buffer (called im2col or im2row), and its initialization is both memory and time-consuming. To overcome this problem, one may use the Indirect Convolution Algorithm. This algorithm replaces the im2row buffer with a much smaller buffer of pointers, called indirection buffer. However, it limits our flexibility in the choice of multiplication micro-kernel, making matrix multiplication slightly less efficient than in the classical GeMM algorithm. To overcome this problem, we propose the Almost Indirect Convolution Algorithm, which initializes small specifically ordered block of values, which is used in matrix multiplication, via indirection buffer, the same way GeMM Algorithms initializes one block from im2row buffer. Our approach allows us to combine computational efficiency and flexibility in shape of GeMM micro-kernels with a small memory footprint of the Indirect Convolution Algorithm. Experiments with convolutions of 8-bit matrices on ARM processors show that our convolution works 14-24% faster than Indirect for a small number of channels and 10-20% faster than classical GeMM-based. This proves that it is perfectly suitable for computing inference of 8-bit quantized networks on mobile devices.

In this work, we consider the pairs generation algorithm based on the distances between elements in metric space. The right generation of training data is an actual issue, and its solution leads to better neural network learning. Understanding the properties of the source data, we can select pairs for training in such a way that the network will pay more attention to elements that are close in the metric space and have different classes. However, the problem arises when these properties are difficult to extract from the data and a more universal pairs generation method is needed. Our method generates pairs using the results of the network from previous iterations, in parallel with the training process itself. Thus, we do not need to evaluate the properties of elements ourselves, and we can use absolutely any data as learning objects. We demonstrate this approach using the example of Korean character recognition, and also compare it with other commonly used pair generation methods.

A machine learning algorithm aims at designing a mathematical model based on a given training data set. Generally, the built model has a set of parameters that need to be adjusted. Since the performance of a given model depends on its settings, the parameters have to be carefully chosen through a fine-tuning step. A good model selection not only boosts performance but also allows a well-generalized model, i.e., a model that works sound on unseen data. In this paper, we assess the effectiveness of some metaheuristic optimization algorithms for support-vector machines (SVM) model selection. Computer simulations show that optimization algorithms that overall outperforms other algorithms using benchmark functions can be, further, definitely used for an efficient SVM model selection for classification. Thus, we show that Teaching–Learning-Based Optimization algorithm is faster and also enables the most accurate classification, even against other proposed methods in the literature for SVM model selection.

For the multi-channel collimators employed in SPECT measurements and in gamma probes (for radio guided surgery) an approach to modeling the functioning of collimator is developed. This approach involves the calculation of point source function and the sensitivity field for the considering device. The effect of finite absorption in collimator material is taken into account. The results of modelling for confocal collimator are presented. The calculations were performed for the attenuation length corresponding to the device 3D-printed from metal and in the assumption of infinite absorption. It makes possible to estimate quantitatively the effect of gamma photons partial propagation through the matrix of collimator on the sensitivity and resolution for the specified material and the geometry of the device.

Automotive augmented reality head-up displays (AR-HUDs) superimpose driving related information with the real world in the direct sight of the driver. A key prerequisite for an immersive AR experience is a highly precise calibration. State-of-the-art methods require large targets and a lot of space in front of the vehicle, or special complex equipment, which is inconvenient in both factories and workshops. In this paper, we propose a low-complexity yet accurate calibration method using only a small sheet of patterned paper as the target, which is laid directly on the windscreen. The full field of view (FOV) can be calibrated, with the optical distortion corrected by extracted warping maps. The changing views of drivers are considered by interpolating both projection parameters and distortion models. The angular reprojection error falls within 0.04°, while the run-time is limited up to 1 minute per viewpoint. Our method shows high applicability in the automotive industry because of both reduced target complexity and competitive reprojection errors. Moreover, due to the reduced effort and simplified equipment, our method opens a way for customers to recalibrate their AR-HUDs themselves.

Material flow detection on conveyor based on machine vision is the research topic of this paper. A belt conveyor system equipped with a camera and micro-controller is used as the test apparatus. The purpose of this experiment is to obtain the quantity of material on the conveyor belt using machine vision and then develop an intelligent speed adjustment system for belt conveyor according to the quantity, so as to avoid waste of energy and reduce the wear of the conveyor. Three image processing algorithms that developed, applied and compared were: 1) Background Subtraction; 2) Canny edge detection and morphological operations; 3) Particle analysis using. It is observed that all three methods perform well for material detection on the conveyor belt. However, the particle analysis method resulted in higher reliability and accuracy with faster processing speed. The research provides new developmental ideas for intelligent conveyor systems.

In this paper, a novel black box adversarial computer vision attack is proposed. The introduced attack is based on removing from images some components described by their Tchebichef discrete orthogonal moments, rather than to perturb them. The contribution of this work is focused on the addition of one more clue, supporting the critical hypothesis that computer vision systems fail because they support their decisions not only in robust features but also in others non-robust ones. In this, context non-robust image features described in terms of Tchebichef moments are excluded from the original images and the approximated reconstructed versions of them are used as adversarial examples in order to attack some popular deep learning models. The experiments justify the effectiveness of the proposed adversarial attack in terms of imperceptibility and recognition error rate of the deep learning classifiers. It is worth noting that the top-1 accuracy of the attacked models was degraded by a factor between 9.48%-70.89% for adversarial images of 65dB to 57dB PSNR values. The corresponding degradation of the top-5 models’ accuracy was between 6.9% and 55.14% for the same quality images. Moreover, the proposed attack seems to have more strength than the Fast Gradient Sign Method (FGSM) attacking method traditionally applying in most cases. These results reveal that the proposed attack is able to exploit the vulnerability of the deep learning models’ towards degrading their generalization abilities. In this paper, a novel black box adversarial computer vision attack is proposed. The introduced attack is based on removing from images some components described by their Tchebichef discrete orthogonal moments, rather than to perturb them. The contribution of this work is focused on the addition of one more clue, supporting the critical hypothesis that computer vision systems fail because they support their decisions not only in robust features but also in others non-robust ones. In this, context non-robust image features described in terms of Tchebichef moments are excluded from the original images and the approximated reconstructed versions of them are used as adversarial examples in order to attack some popular deep learning models. The experiments justify the effectiveness of the proposed adversarial attack in terms of imperceptibility and recognition error rate of the deep learning classifiers. It is worth noting that the top-1 accuracy of the attacked models was degraded by a factor between 9.48%-70.89% for adversarial images of 65dB to 57dB PSNR values. The corresponding degradation of the top-5 models’ accuracy was between 6.9% and 55.14% for the same quality images. Moreover, the proposed attack seems to have more strength than the Fast Gradient Sign Method (FGSM) attacking method traditionally applying in most cases. These results reveal that the proposed attack is able to exploit the vulnerability of the deep learning models’ towards degrading their generalization abilities.

Automated Deception Detection (ADD) is a challenging task and still under study as a visual analysis task. Based on the idea that human micro-expressions and body movements could be used as clues for ADD, many works have proposed some action recognition models for extracting face and body spatiotemporal features. However, these features are not sufficient evidence for deception; moreover, micro-expressions are difficult to detect and real-life deception samples are hard to collect, thus ADD still has many challenges. In this paper, we present a global two-stream network (GTSN), which not only extracts face and body features, but also utilizes the correlation between the deceptions. GTSN can improve the accuracy of deception detection by adding historical information based on the correlation between the deceptions. We build a dataset named Deception-Truthful (DT) for evaluating the performance of our proposed model. Experimental results demonstrate that our GTSN model outperforms other action recognition models used for ADD. Further, the proposed GTSN model also performs well on the real trial videos widely used in ADD.

One of the business tasks of personal documents recognition using mobile devices is to obtain a high quality image of document owner’s photograph. Such photographs are used to verify and identify the owner of the document. For example, in remote self-service systems, the image of a photo can be compared to a selfie. When a document is captured with a mobile device camera in uncontrolled conditions, the photograph’s image quality varies greatly from frame to frame. In this paper, factors influencing the image quality of a photograph are considered: features of personal documents, capture and recognition processes. A method for choosing the best photograph image is proposed. The quality of the method is assessed on real data by the method of stochastic modeling.

We study the effects of the additional input to deep multi-view stereo methods in the form of low-quality sensor depth. We modify two state-of-the-art deep multi-view stereo methods for using with the input depth. We show that the additional input depth may improve the quality of deep multi-view stereo.

Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use and one of the reasons for that is poor transferability of the models to data from different sources or acquisition domains. Development of new methods and algorithms for the transfer of training and adaptation of the domain in multi-modal medical imaging data is crucial for the development of accurate models and their use in clinics. In present work, we overview methods used to tackle the domain shift problem in machine learning and computer vision. The algorithms discussed in this survey include advanced data processing, model architecture enhancing and featured training, as well as predicting in domain invariant latent space. The application of the autoencoding neural networks and their domain-invariant variations are heavily discussed in a survey. We observe the latest methods applied to the magnetic resonance imaging (MRI) data analysis and conclude on their performance as well as propose directions for further research.

The study is referred to a task of 2D data slope estimation. We consider the integral projections analysis technique and a common criterion of sum of squared values (SSV) for optimal angle detection. This criterion is dependent on the density of input data and for very sparse data its efficiency significantly decreases. We propose the alternative criteria – the sum of the inversed lengths (SIL) that preserves SSV characteristics for dense data but that is much more robust for sparse input. The experiments conducted on simulated and real datasets demonstrate better quality of slope detection using the proposed criterion.

Most of current methods for fatigue detection are based on fatigue features extracted from face or head. This paper proposes a novel fatigue detection mode based on non-contact respiratory detection. This paper proposes to use the respiratory rate feature for fatigue detection. Since the appearance of the respiratory rate fatigue feature will be earlier than the facial fatigue feature, fatigue can be predicted early by the respiratory rate. The difficulty of respiratory rate fatigue detection lies in the extraction of respiratory rate features and detection accuracy, so this paper adopts the non-contact respiratory detection based on dense optical flow method to solve the above problems. In this paper, a large number of experimental comparisons between respiratory rate fatigue detection and facial fatigue detection have confirmed that fatigue can be detected faster through respiratory rate fatigue characteristics, but there will still be a small deviation in the detection of respiratory rate, so this paper proposes to compare facial fatigue characteristics with breathing The fatigue detection method combined with the rate of fatigue characteristics, this method greatly improves the accuracy and speed of fatigue detection.

Encapsulated inorganic particles with high melting points (>300°C) are desired as high-temperature Phase Change Materials (PCMs) for next-generation Latent Heat Thermal Energy Storage (LHTES) systems. One of the many challenges during the development of PCMs is to achieve a high throughput that in turn depends on accurately modeling the relation between process parameters and geometric and thermal properties of the PCMs particle. During the production of the PCMs, a high-speed infrared camera is used to acquire images of the encapsulated material under controlled illumination conditions. This research article focuses on the development of image processing techniques for both geometric and thermal feature extraction during the development of the PCMs. A user-friendly GUI has been designed in MATLAB and preliminary experimental results have demonstrated that the method is fast, accurate and reliable for a high throughput production. The extracted features will be used to develop Machine Learning (ML) models to predict the geometric and thermal properties of the PCM based on the process parameter settings. The ML model will accelerate the search for the optimized process settings to boost the throughput of the production.

In that paper, we a suggest lightweight filtering neural network, which implements the filtering stage in the Filtered Back-Projection algorithm (FBP), but good reconstruction results are achieved not only in ideal data but also in noisy data, which a usual FBP algorithm cannot achieve. Thus, our neural network is not an only variation of Ramp filter, which is usually used then FBP algorithm, but also a denoising filter. The neural network architecture was inspired with the idea of the possibility of the Ramp filtering operation’s approximation with sufficient accuracy. The efficiency of our network was shown on the synthetic data, which imitate tomographic projections collected with low exposition. In the generation of synthetic data, we have taken into account the quantum nature of X-ray radiation, exposition time of one frame, and non-linear detector response. The FBP reconstruction time with our neural network was 13 times faster than the time of reconstruction neural network from Learned Primal-Dual Reconstruction, and our reconstruction quality 0.906 by SSIM metric, which is enough to identify most significant objects.

We propose a neural network-based solution for three different tracks of 2nd International Illumination Estimation Challenge (chromaticity.iitp.ru). Our method is built on pre-trained Squeeze-Net backbone, differential 2D chroma histogram layer and a shallow MLP utilizing Exif information. By combining semantic feature, color feature and Exif metadata, the resulting method – SDE-AWB – obtains 1st place in both indoor and two-illuminant tracks and 2nd place in general track.

In the field of 3D digitization of real objects using modern scanning devices, dense point clouds can be obtained. This data point can have redundancy. To solve this problem, we present a new simplification method based on clustering and Shannon's entropy. This approach optimizes the number of 3D point clouds by keeping the original point cloud characteristics. To show the robustness of the technique, we have applied it on different point cloud and making comparisons with other methods. It can be said, according to the obtained results, that our method is effective.

When taking photos of historical buildings and landmarks, one can often encounter more modern and less attractive objects blocking the view (poles, electricity cables, road signs, and others). Photographers strive to take the unobstructed shot, without random objects popping up in the image. However, they avoid deleting unwanted objects with inpainting or exemplar-based methods, because it can introduce notable artefacts. We propose a new algorithm for removing large static objects such as road signs, sitting people, and parked cars from a photograph keeping its originality and unique details. We do not use artificial patterns to fill covered regions, only two shots taken from different angles. Usually, removing long thin objects such as road sign poles cause image deformations, but our approach avoids this. For comparison, we created a dataset from “Caltech Buildings” and “Kaggle Architecture” Datasets and added road signs, cars, and other objects to photos. We compared our approach with state-of-the-art methods such as Deep Image Prior, Gated Convolution, and the Region Filling by block sampling. The real photographs of historical building demonstrate the effectiveness of our algorithm. Code and example images are available on GitHub¹.

Deep neural networks are widely used in various AI systems. Many such systems rely on the edge computing concept and try to perform computations on end devices while still being energy and memory efficient. Therefore, substantial time and memory requirements are imposed on neural networks. One way to improve neural network efficiency is to simplify computations inside a neuron. A bipolar morphological neuron uses only addition, subtraction, and maximum operations inside the neuron and exponent and logarithm as activation functions for the network layers. These operations allow fast and compact gate implementation for FPGA and ASIC. In the paper, we consider the usage of bipolar morphological (BM) networks for document binarization. We examine the DIBCO 2017 binarization challenge and train the bipolar morphological convolutional neural network of U-Net architecture. Despite some accuracy decrease for a model with all BM convolutional layers, one can flexibly control the accuracy by using the partially converted model. It should be noted that even the fully BM model is suitable for solving the problem in practice.

Recovering three-dimensional structure from images is a long-standing ill-posed inverse problem in computer vision. This paper presents a simple and highly scalable method to reconstruct dense 3D point cloud from multi-view images by estimating per-pixel depth using an evolutionary computation technique – CMA-ES. The proposed method uses ZNCCbased template matching to reconstruct fine details of textured regions and DAISY-based feature matching to reconstruct smooth surface of homogeneous regions. We handle the problem of reconstructing large homogeneous regions using distance transform-based adaptive median filtering. The proposed method is highly scalable since pixels are processed independently at all stages of reconstruction – depth map estimation, refinement, and fusion. This enables the proposed method to be parallelized at the pixel-level, unlike most existing methods that can only be parallelized at the image-level. Experimental results on Middlebury benchmark dataset demonstrate the robustness and efficacy of the proposed method in reconstructing textured as well as homogeneous regions.

Image recognition includes problems where special features can be found only in a specific area of an image. This fact suggests us to apply different filters to different areas of input images. Convolutional networks have only fully-connected and locally-connected layers to make it. A Fully-connected layer erases the position factor for every output and a locally connected layer storage an enormous number of parameters. We need a layer that can apply different convolution kernels for different areas of an input image and not carry so many parameters as a locally-connected layer for high scale resolution images. This is why in this paper, we introduce a new type of convolutional layer - a block layer, and a way to construct a neural network using block convolutional layers to achieve better performance in the image classification problem. The influence of block layers on the quality of the neural network classifier is shown in this paper. We also provide a comparison with neural network architecture LeNet-5 as a baseline. The research was conducted on open datasets: MNIST, CIFAR-10, Fashion MNIST. The results of our research prove that this layer can increase the accuracy of neural network classifiers without increasing the number of operations for the neural network.

Despite significant progress in computer vision, pattern recognition, and image analysis, artifacts in imaging still hampers the progress in many scientific fields relying on the results of image analysis. We here present an advanced image-based artifacts suppression algorithm for high-resolution tomography. The algorithm is based on guided filtering of a reconstructed image mapped from the Cartesian to the polar coordinates space. This postprocessing method efficiently reduces both ring- and radial streak artifacts in a reconstructed image. Radial streak artifacts can appear in tomography with an off-center rotation of a large object over 360 degrees used to increase the reconstruction field of view. We successfully applied the developed algorithm for improving x-ray phase-contrast images of human post-mortem pineal gland and olfactory bulbs.

In this paper, we proposed an unsupervised learning method for estimating the optical flow between video frames, especially to solve the occlusion problem. Occlusion is caused by the movement of an object or the movement of the camera, defined as when certain pixels are visible in one video frame but not in adjacent frames. Due to the lack of pixel correspondence between frames in the occluded area, incorrect photometric loss calculation can mislead the optical flow training process. In the video sequence, we found that the occlusion in the forward (t→t+1) and backward (t→t-1) frame pairs are usually complementary. That is, pixels that are occluded in subsequent frames are often not occluded in the previous frame and vice versa. Therefore, by using this complementarity, a new weighted loss is proposed to solve the occlusion problem. Our method achieves competitive optical flow accuracy compared to the baseline and some supervised methods on KITTI and Sintel benchmarks.

The Diabetic Retinopathy (DR) is a worldwide eye disease that causes visual damages and can leads to blindness. Therefore, the detection of the DR in the early stages is highly recommended. However, a delay is registered for ensuring early DR diagnosis which caused by the low-rate of the ophthalmologists, the deficiency of diagnosis equipment and the lack of mobility of elderly patients. In this paper, the main objective is to provide a mobile-aided screening system of moderate DR. Within this aim, we propose a classifier-based method which is based on detecting the Hard Exudate (HE) lesions that occur in moderate DR stage. A set of features are extracted to ensure an accurate and robust detection with respect to modest quality of fundus images. Moreover, the detection is provided in a low complexity processing to be suitable for mobile device. The aimed system corresponds to the implementation of the method on a smartphone associated to an optical lens for capturing fundus image. The system reached satisfactory screening performance where an accuracy of 98.36%, a sensitivity of 100% and specificity of 96.45% are registered using the DIARETDB1 fundus image databases. Moreover, the screening is performed in an average execution time of 2.68 seconds.

In this paper we explore a set of modifications of the cascade structure of the Viola-Jones detector on the example of solving stamp detection problem. The experiments on the public “SPODS” dataset for various document attributes extraction problems with extremely limited training set are presented. The positive training set is augmented by applying various image processing algorithms relevant to the stamp model to an available in a single instance image for each stamp type. We describe and analyze such structures of the Viola and Jones classifiers as the original cascade structure, tree, soft cascade, and perform the training experiments. Experimental results show that each modification of the cascade structure of the classifier has its own advantages and disadvantages, and the choice of the Viola-Jones classifier design significantly affects the quality of solving object detection problem.

The growing usage of smart mobile devices have made authentication with biometric data more convenient. On the other side, videos and photos of users are becoming more available online. This makes it easier for attackers to spoof the authentication systems which rely on face and eye-region data for instance. One major problem with current Presentation Attack Detection (PAD) systems is their lack of generalization to data captured by different sensors or in different environments. In this paper, we propose the use of unsupervised domain adaptation to solve this PAD problem, specifically the iris PAD. Our model is composed of symmetric classifiers and two per-class domain discriminators. Interaction between class probabilities and domain classification is utilized to jointly adversarialy train a mobile-oriented feature extraction network, capable of generating domain-invariant features. The approach is evaluated on three benchmark iris PAD datasets. Results show up to 40% improvement in cross-dataset Average Classification Error Rate (ACER) proving the effectiveness of the approach in increasing the robustness and generalization of biometric PAD systems.

Part semantic segmentation based on deep learning provides a new insight for accurate vision understanding of noncooperative satellite as well as for further on-orbit servicing tasks like inspection, repair, and close-proximity robotic manipulation. However, carrying out such researches requires a tremendous amount of data, which is extremely hard and expensive in space. Moreover, the manual annotation for fine-grained tasks like segmentation will cost a lot of labor. Thus, in this paper, we present an efficient method of automated synthetic datasets construction for part-level segmentation of non-cooperative satellite, which is capable of generating thousands of multi-source data (RGB image and point cloud) and the corresponding high-quality annotation. Specifically, the Fibonacci lattice is used for multiple viewpoints sampling of the virtual camera to capture RGB-D images. A trick of segmentation of the customized image in HSV color space is applied to get labels automatically. Furthermore, we employ several data augmentation techniques to expand and diversify the datasets, which improves the generalization of the algorithm. Finally, we carry out the case study using the pointnet++ network based on our generated point cloud data, to validate the feasibility and effectiveness of our method.

In this work we study the effect of activation functions in a neural network. We consider how activation functions with different properties and their combination affect the final quality of the model. Due to optimization and speed performance issues with most of bounded functions that are represented by sigmoids, we propose the generalized version of SoftSign function - ratio function (rf). Its shape greatly depends on introduced degree parameter, which in theory leads to new interesting property - contraction to zero. For evaluation, we chose image binarization problem: based on UNet architecture of DIBCO-2017 winners, we conducted all experiments with replacing activation functions only. Our research has led us to the state-of-the-art results in binarization quality on DIBCO-2017 test dataset. U-Net with modified activation functions significantly outperforms all existing solutions in all metrics.

Retinal blood vessel segmentation in fundus images has become essential for various applications of computer-aided anomaly analysis. In this work, we propose an automated segmentation method based on mathematical morphology combined with entropy information, what allows an accurate classification of each pixel depending on its neighbors’ comportment. The main contribution resides in the joint integration, for the first time within the context of retinal blood vessel segmentation to the best of our knowledge, of the entropy information within the hysteresis thresholding. The first step of the method consists of classifying the image pixels using morphological operators. Then, we extract the entropy information followed by a hysteresis thresholding in order to isolate the retinal blood vessels from the background, while ensuring the smoothness and the spatial coherence of the kept pixels. Finally, morphological operators are applied to refine the segmentation results. Qualitative and quantitative tests were performed on the standard DRIVE and STARE datasets and the obtained results proved the effectiveness of the proposed method.

The growing popularity of mobile services increases the risks of financial and other losses from fraudulent user actions. To reduce the number of illegal actions and comply with the law when using mobile services, it is often scheme that user presents it’s identity document. In case of remote access via a mobile device, this means receiving and analyzing a video of a document image. One of the criteria for the authenticity of the captured security document is the presence of security optically variable devices (kinegrams, holograms). Reliable determination of the presence or absence of such security elements from video stream frames is greatly complicated by changes in color between frames. The paper discusses the possibility of using a priori information about the monochrome photograph of the document owner to compensate changes in color between frames. Color distributions are investigated on the example of black-and-white photographs. A new method for automatic white balance correction is proposed. The results of the method are tested on real data obtained with a mobile device.

In this paper, we propose a deep learning instance segmentation framework for particle extraction of microscopic images that aims at calculating planktonic species distribution and concentration in-situ. The framework comprises three essential functional tasks on in-situ time-series images collected from an autonomous underwater vehicle: 1) manual labeling of the captured images, 2) object localization, segmentation, and identification, and 3) class distribution and planktonic organisms concentration calculation. Our proposed framework is based on the mask R-CNN architecture provided by the Detectron2 library developed by Facebook Artificial Intelligence Research (FAIR) for instance segmentation. Due to its modular design, we compare the performance of different networks by alternating the backbone sub-network in order to choose the most suitable architecture for the task of instance and semantic segmentation. We compile a custom annotated dataset from planktonic time-series images and train the different models over this dataset to perform the instance semantic segmentation. Evaluation results of the proposed framework, utilizing the best performing deep learning architecture along with the new annotated dataset, show better performance in terms of speed and accuracy of both in-situ segmentation and classification compared to traditional segmentation methods. In addition, we observe a significant improvement in the object classification quality when we train the model over our newly annotated dataset instead of training it over the dataset generated from the traditional methods. The inferred data from our novel instance segmentation framework, which provides the particle class distribution and concentration, can then be used to assist in constructing a dynamic probability density map of planktonic communities dispersion and abundance.

User Experience Design (UX Design) comes from focusing on how products, in reality, affect the user's experience. In particular, the design of multi-modal interfaces for blind people facilitates the flexible and natural product or service capacity and improves blind people's interaction by overcoming the various existing constraints associated with any particular interaction. There have been various attempts to help visually impaired people appreciation of visual artwork, including multi-modal associations. However, these methods can only provide general information in terms of edge and pattern recognition by the sense of touch and restrained by the availability and number of specially developed artworks. We propose a novel method explaining visual artworks through image caption generation using artificial intelligence (AI) to improve artwork accessibility. This method can objectively describe any impressionism artwork used as a standalone description of art interpretation for blind people or can aide tactile-based methods. Based on end-to-end learning with a deep neural network, an encoder-decoder architecture model is adopted, and comprehensive experiments perform to confirm the stability of generated image captioning for stylized MS-COCO datasets with impressionism.

The development of methods for improving the quality of tomographic images is an urgent task, as the presence of artifacts and insufficient sharpness of tomography result can cause erroneous decisions in medical diagnostics, when analyzing the structure of geological cores, etc. It is assumed that several artifacts arise due to inadequate a priori information involved in constructing the result of tomography, and a two-stage method is proposed for constructing an estimate of the distribution of the absorption coefficient of the sample. At the first stage, without using a priori information about the internal structure, the methods of the theory of computer-aided measuring systems construct an estimate of the absorption coefficient, which results in a blurred image of the internal structure of the object. At the second stage, the resolution of this image is increased by the method of moving averages with coefficients calculated from the condition of maximum accuracy for estimating the brightness of the central pixel of the window. Further, this estimate is refined from the natural conditions of the non-negativeness of the estimated brightness. Examples of the application of the method for assessing the structure of a child's tooth are given.

Image mosaicing is a challenging problem when there is one (or more) corrupt image(s) in the input sequence. Since the transformation of later images rely on the previous transformation calculations, one miscalculation error caused by the corrupt image propagates to other image transformations, which fails the whole process. It is not a trivial task to detect and remove these corrupt images, which we call “outliers”. In this paper, we propose a dynamic programming inspired outlier rejection algorithm to identify and remove the corrupt image(s) from the sequence. Our approach stores the previously calculated transformation matrices in a 2D array and determines the validity of composite transformations based on a decision criterion. This criterion identifies the transformations coming from corrupt images by comparing the direct registration of images and the composite transformations from existing matrices in the array. We have performed experiments on both synthetic and real datasets. Visual and numerical results show that the proposed algorithm is an efficient tool for detecting and rejecting the outlier image from the mosaic image.

Generative adversarial networks (GANs) have been successfully applied to transfer visual attributes in many domains, including that of human face images. This success is partly attributable to the facts that human faces have similar shapes and the positions of eyes, noses, and mouths are fixed among different people. Attribute transfer is more challenging when the source and target domain share different shapes. In this paper, we introduce a shape-aware GAN model that is able to preserve shape when transferring attributes, and propose its application to some real-world domains. Compared to other state-of-art GANs-based image-to-image translation models, the model we propose is able to generate more visually appealing results while maintaining the quality of results from transfer learning.

In Computed tomography (CT) usage of common reconstruction algorithms to the projection data acquired with polychromatic probing radiation leads to the appearance of a cup-like distortion. CT image quality can be improved by adjusting the CT scanner or the reconstruction algorithm, but for this purpose assessment of cupping artifacts evaluation needs to be done. Existing assessment methods either rely on expert opinion or require an object binary mask, which can be unavailable. In this paper, we propose a method for blind assessment of cupping artifacts that do not require any prior information. The main idea of the proposed method is to evaluate the degree of change in intensity near automatically found edges of optically dense objects. We prove the applicability of the method on the collected dataset with cupping artifacts. The results show a monotonic dependency between the severity of cupping artifacts and the calculated with the proposed method value.

Modern vehicles include a vast number of intellectual functions such as lane-keeping assist (LKA), vehicle, pedestrian and obstacle recognition (FCW, PPS) which are implemented in the advanced driver assistance system (ADAS). These functions allow a vehicle to localize itself correctly within the road lane and to increase the overall system safety. It is also critical for vehicle motion and planning the target trajectory. Previously, algorithms implementing ADAS functions were based on classical computer vision approaches (e.g. edge detection, morphology, Hough transform), which did well only on a rather simple road scene. Modern state-of-the-art systems are based on semantic segmentation networks, it is the unquestionable trend. With a more thorough study of the road scene segmentation issues we face the problem that the existing benchmark suites such as MOTS, KITTI as well as recent DNNs for the road/lane semantic segmentation employ only mutually exclusive classes i.e. in this case, a pixel can belong to a single class only. But if we recognize the road scene, a pixel can easily refer to several classes, e.g. to ego-lane and crosswalk. The classical approach with mutually exclusive classes will give preference to only one class in this case and we will get an ego-lane consisting of two components. As a result, it may be difficult to restore the ego-lane at the stage of post-processing, see Figure 1. To overcome this problem, in the paper we propose the approach with multiple segmentation maps as an output of the DNN architecture, as well as a multi-map loss function. In this case, each pixel is referred to several classes at the same time (depending on the number of layers) and we don’t have the restriction to use mutually exclusive classes. The DNN classifier for each segmentation map has a separate activation branch and loss function.

This paper introduces a novel dataset FeatherV1, containing 28,272 images of feathers categorized by 595 bird species. It was created to perform taxonomic identification of bird species by a single feather, which can be applied in amateur and professional ornithology. FeatherV1 is the first publicly available bird's plumage dataset for machine learning, and it can raise interest for a new task in fine-grained visual recognition domain. The latest version of the dataset can be downloaded at https://github.com/feathers-dataset/feathersv1-dataset. We also present feathers classification task results. We selected several deep learning architectures (DenseNet based) for categorical crossentropy values comparison on the provided dataset.

Incomplete Point clouds obtained from one-side scanning always result in structural loss in 3D shape representations, thus many learning-based methods are proposed to restore complete point clouds from partial ones. However, most of them only utilize global features of inputs to generate outputs, which might lose details. In this paper, a new method that utilizes both global and local features is proposed. First, Local features are extracted from inputs and analyzed under the conditions interpreted by global features. Second, conditional local feature vectors are deeply fused with each other via graph convolution and self-attention. Third, deeply-fused features are decoded for generating coarse point clouds. Last, global features extracted from inputs and coarse outputs are combined to generate fine outputs with high-density. Our network is trained and tested on eight categories of objects in ModelNet. The results show that our network is able to overcome instability in local feature awareness, restore complete point clouds with more details and smoother shapes, and outperform most of those existing methods both intuitively and quantitatively. Our source codes will be available at: https://github.com/wuhang100/LRA-Net.

This article is focused on methods of search for falsifications in scanned copies of business documents. This task arises from a comparison of two copies of business documents signed by two parties. The comparison should be performed to detect possible changes made by one of the parties. This problem is relevant, for instance, in the banking sector when signing agreements on paper. The method of partial search for matching flexible documents, where text attributes may be changed, and unintentional modifications of non-essential words may be made is considered. The method of comparison of two scanned images based on the recognition and analysis of N-grams word sequences is proposed. The proposed method has been tested on private dataset. The proposed method has demonstrated high quality and reliability of the search for differences in two samples of one agreement-type document.

Line detection is an important computer vision task traditionally solved by Hough Transform. With the advance of deep learning, however, trainable approaches to line detection became popular. In this paper we propose a lightweight CNN for line detection with an embedded parameter-free Hough layer, which allows the network neurons to have global strip-like receptive fields. We argue that traditional convolutional networks have two inherent problems when applied to the task of line detection and show how insertion of a Hough layer into the network solves them. Additionally, we point out some major inconsistencies in the current datasets used for line detection.

Automatic violence detection is one of the most interesting tasks in video surveillance applications. It is essential to ensure human safety by preventing fatal accidents. In this paper, we propose a spatiotemporal feature based on convolutional neural network and optical flow information for violence detection. Our descriptor, named distribution-based CNN feature (DCNN), estimates first the joint distribution of the optical flow magnitude and orientation around STIP points in the aim of modeling the motion structure. Then, it extracts deep features using the pretrained ResNet50 network. Our DCNN feature is finally fed into the SVM classifier for training. We evaluate the proposed descriptor on two challenging benchmark datasets designed for violence detection in both uncrowded and crowded scenes. For the two datasets, our DCNN feature shows significant improvement compared to the state-of-the-art descriptors.

Research and development in agricultural robots are continuously increasing. However, dynamically changing agricultural environments provide adverse conditions to robotics operability. In order to perform the agricultural tasks safely and accurately, reliable landmarks from the surrounding environment need to be identified. In this work, deep learning is employed for accurate and fast detection of high-level features of vineyards, the vine trunks. More specifically, Faster regions-convolutional neural network (Faster R-CNN), You Only Look Once version 3 (YOLOv3) and YOLOv5 are tested for real-time vine trunk detection. The models are trained with an in-house dataset designed for the needs of this study, containing 1927 annotated vine trunks in 899 different images. Comparative results indicate YOLOv5 as the configuration that allows the faster and most accurate vine trunk detection, achieving an overall Average Precision of 73.2% in 29.6 ms. The high precision combined with the fast runtime performance prove that the YOLOv5 detector is suitable for real-time vine trunk detection executed by an autonomous harvesting robot.

White balance estimation (WBE) is one of the most fundamental and crucial steps in modern Image Signal Processor (ISP). Recent years have witnessed the advancements of deep-learning based WBE. However, existing models were mostly trained on individual datasets with limited samples captured using various camera sensors, making it hard for model generalization. In this paper, we propose a novel Channel-Attention optimized U-net model, in which an angular loss is embedded, to accurately estimate the white balance. We demonstrate our approach on recently released largescale dataset “Cube Plus” captured using the same camera sensor, offering state-of-the-art performance.

In this work, we consider a problem of quadrilateral document borders detection in images captured by a mobile device’s camera. State-of-the-art algorithms for the quadrilateral document borders detection are not designed for cases when one of the document borders is either completely out of the frame, obscured, or of low contrast. We propose the algorithm which correctly processes the image in such cases. It is built on the classical contour-based algorithm. We modify the latter using the document’s aspect ratio which is known a priori. We demonstrate that this modification reduces the number of incorrect detections by 34% on an open dataset MIDV-500.

In this paper we explore possibilities of memory cost reduction without significant loss of classification accuracy in connection with the problem of the ID document type recognition on mobile devices. The studied classic approach is based on representing images using constellation of feature points and descriptors. The distortion parameters are estimated by applying RANSAC. Experimental data details the approach limitations (memory, speed and accuracy) in dependence of the descriptor type. In order to maintain accuracy when using low dimensional descriptors we suggest to modify the basic approach using additional features characteristic of the document such as straight lines and quadrangles. In addition, an early filtration of the samples and the hypotheses used in RANSAC. It was shown that the proposed modifications have a positive contribution for all types of descriptors considered. The suggested algorithm was tested using the open dataset MIDV-500. The modified approach allows to achieve an accuracy improvement and significant speed up of distortion parameters estimation in RANSAC. It was shown that using compact descriptors in conjunction with the presented method allows reduce required memory cost by more than 7 times with near-zero (0.2%) loss of accuracy, and more than 14 times with the loss of accuracy is about 18%.

The paper proposes an approach for matching of digitized copies of business documents. This task arises when comparing two versions of the same document, genuine and forgery, to find possible modifications, for example in the banking sector during the conclusion of contracts in paper form to avoid possible fraud. The matching method of two documents based on comparison images of text lines using Variational Autoencoder (VAE) trained on genuine images and calculation Fisher information metric to find modifications. Experiments were conducted on the public Payslips dataset (in French). The results show the high quality and reliability of finding document forgeries and are compared to the results of the method which applies OCR and image matching.

The paper presents an algorithm for constructing a local path for a vehicle with nonholonomic kinematics of an automobile type. A local path is a sequence of transitions in the graph of possible maneuvers that minimizes a given cost function. The graph is constructed by duplicating along the global path pre-calculated in a curvilinear coordinate system set of kinematically feasible motion primitives. The use of pre-computed motion primitives significantly reduces the time of graph construction. The weight of each maneuver – the edge of the transition graph – is calculated as a weighted sum of costs based on several criteria. The specified cost function minimizes maneuvering and maintains a safe distance to static obstacles. The information about obstacles is extracted from an occupancy grid map. Dijkstra‘s algorithm is used to search a path in the weighted directed graph. The algorithm was tested on a dataset containing real road scenes. Each scene represents a given global path and a static environment model where a safe local path must be found. Local path search is performed in real-time. Experiments have shown that safe local paths have been found in all scenes where it was physically possible. At the same time, the obtained local paths were on average only on 1:3% longer than the given global paths which demonstrate the high applicability of the proposed algorithm.

Microtomography is a powerful method of materials investigation. It enables to obtain physical properties of porous media non-destructively that is useful in studies. One of the application ways is a calculation of porosity, pore sizes, surface area, and other parameters of metal-ceramic (cermet) membranes which are widely spread in the filtration industry. The microtomography approach is efficient because all of those parameters are calculated simultaneously in contrast to the conventional techniques. Nevertheless, the calculations on Micro-CT reconstructed images appear to be time-consuming, consequently representative volume element should be chosen to speed them up. This research sheds light on representative elementary volume identification without consideration of any physical parameters such as porosity, etc. Thus, the volume element could be found even in noised and grayscale images. The proposed method is flexible and does not overestimate the volume size in the case of anisotropic samples. The obtained volume element could be used for computations of the domain’s physical characteristics if the image is filtered and binarized, or for selections of optimal filtering parameters for denoising procedure.

We describe the technique used to train and customize deep learning models to detect, track, and identify soccer players, who are recorded during soccer games using custom camera settings. The player detection model is customized to allow the detection of person class objects from video input. Two newly developed filters, spatial feature filters, and bounding box location filters have described that help in classifying players and audiences. A new tacking paradigm is illustrated to generate tracks of soccer players with fewer swaps, thereby reducing efforts of human annotators in later stages. A new method of identifying every player by detecting player t-shirt numbers has been developed and illustrated. This method provides tracks with high confidence and identity to most of the player corresponding to individual t-shirt number. Finally, we provide a unique result assessment technique to judge the performance of the complete model.

In the field of document analysis and recognition using mobile devices for capturing, and the field of object recognition in a video stream, it is important to be able to combine the information received from different frames, since the quality of text recognition depends on the effectiveness of collecting the maximal amount of information about the target object. This paper examines and compares the effectiveness of two different combination approaches, namely pre-combination of images before recognition and the combination of recognition results. The combination methods are briefly described. The quality of the combined results obtained using different methods was measured and compared on the MIDV-500 dataset. The results show that the approach with a combination of text strings recognition results is more effective in comparison with the preliminary combination of images. It can be concluded that simple image stacking with projective alignment does not allow to achieve a comparable recognition results combination quality, and thus in order to include the information about per-frame changes of the text images more sophisticated image combination algorithms need to be employed.

One of the most important problem in constructing computer vision systems for embedded and mobile devices is offline recognition of text strings. In this paper, we analyze the problem of text strings recognition process in a video stream using best frame selection. This method allows to incorporate the information from multiple views of the same target object, thus increasing the overall extraction accuracy. A stopping method is proposed, which allows to make an automatic stopping decision, i.e. to terminate the process at the optimal time in order to maximize the responsiveness of the system. Experimental evaluation on open identity document datasets MIDV-500 and MIDV-2019 show that the proposed stopping rule allows to decrease mean error level of the text recognition results in comparison with a baseline approach which stops after a fixed amount of processed frames.

Two-dimensional barcode reading is still a challenging problem in the computer vision domain. To simplify this reading process specially designed synchronization patterns are introduced into the barcode structure. Their precise localization allows to use block-based matrix extraction strategy, which is essential for processing images captured in an uncontrolled environment. This paper describes an algorithm for precise localization of such synchronization patterns for the Aztec Code symbology. The proposed algorithm is based on uniform linear motion model usage supported by Kalman filtering to increase its estimation accuracy. Experimental results showed that the replacement of basic ISO-based algorithm with the proposed algorithm increased reading quality from 39% to 98% on the dataset which consists of 1628 real-life document images captured with mobile device cameras in uncontrolled conditions.

Traditional steel pipe counting can only be performed manually, which has the problems of large workload, error prone and low efficiency. This paper proposes a steel pipe counting system based on image recognition. First of all, the steel pipe image is collected from the camera, and then the digital image processing technology is used to perform image enhancement, edge detection and morphological operation and other preprocessing, then the steel pipe is identified and counted by the Hough circle transformation, and finally the steel pipe automatic is developed using VC++ software Technical software. Experiments show that the automatic counting accuracy and efficiency of steel pipes in this system are high.

Barcode recognition via the mobile device camera is an actual problem which occurs in various fields. To receive experimental baselines and provide methods comparison researchers require some carefully annotated data samples. For those purposes, some datasets have already been collected and published in the public domain. But they suffer from different disadvantages. In this paper, we present a novel challenging dataset designed for barcode reading quality evaluation. It is populated using the generative approach with well-established image augmentation technique application. Among them, different geometrical transformations, kind of noises, brightness and lighting variance were used. The generative approach also allowed to populate ground-truth automatically and exclude errors introduced during the manual process. The dataset consists of training (41184 images) and validation parts (10296 images). It contains seven most popular symbologies: CODABAR, CODE-39, CODE-93, CODE-128, EAN-13, UPC-A, UPC-E. As an experimental baseline, the reading quality obtained with the open-source Zxing library is provided. The dataset with all the supplementary materials is available at ftp://smartengines.com/barcode.

This paper proposes several pre-processing algorithms to improve facial expression recognition based on Convolutional Neural Networks (CNNs) models. The proposed CNN model was trained on the Extended Cohn-Kanade dataset (CK+) after applying the pre-processing stages and achieved competitive results (93.90% recognition accuracy) despite its simple and light architecture. Using this CNN model, a study on the impact of each pre-processing algorithm when extracting facial features is presented. In the end, it is understood that pre-processing algorithms help CNNs to extract the most relevant features for each facial expression more effectively, reducing the overfitting and increasing the recognition accuracy. Attention maps before and after the pre-processing step are shown in order to visualize its impact when the proposed CNN model makes a prediction.

For some years now, the idea of a fully autonomous driving car has been monopolizing the attention of the entire automotive sector. The main motivation behind all this interest is that autonomous driving cars could potentially improve road safety simply by subtracting human error, which causes thousands of road fatalities worldwide every year. One of the biggest challenges, for which researchers are working hard to give answers, concerns the search of solutions to improve the people’s response to this vehicles once they are put on the road: an incorrect answer can lead to their unsuccess, with a great monetary loss for all the companies that have invested in these technologies. Recently, we worked on a project that went in this direction and which regarded the possibility to change the driving style of the autonomous driving cars based on the passengers’ facial expressions. This is particularly useful in the event that they experience fear: the car may react to these feelings by adopting a careful driving style and stopping if these feelings do not improve within a certain period. In this paper, we want to discuss about the improvements we have made to this project.

Camera-based road scene analysis is an important task for building driving assistance systems and autonomous vehicles. An crucial component of road scene analysis is detection, tracking, and recognition of text object. In this paper, we consider the recognition of road scene text objects in sequences of video frames, and propose an approach to per-frame recognition results accumulation with a dynamic stopping decision. Experimental evaluation on an open dataset RoadText-1K showed that the proposed approach allows to achieve mean lower recognition error for the same mean number of processed frames, and significantly reduce the number of text objects which have to be recognized in each frame, thus relieving the load on the computational unit.

Computer Vision has been applied to augment traditional board games such as Chess for a number of reasons. While augmented reality enhances the gaming experience, the required additional hardware (e.g. head gear) is still not widely accepted in everyday leisure activities, and therefore, camera based methods have been developed to interface the computer with the real-life chess board. However, traditional 2D camera approaches suffer from ill-defined environmental conditions (lighting, viewing angle) and are therefore severely limited in their application. To answer this issue, we have incorporated a consumer-grade depth camera based on patterned illumination. We could show that in combination with traditional 2D color images, the recognition of chess pieces is made easier, which allows seamless integration of the real-life chess pieces with the computer program. Our method uses a fusion approach from depth and RGB camera data and is suitable for two distant players to play against each other, using two physical sets of chess.

Although gesture recognition has been intensely studied for decades, it is still a challenging research topic due to difficulties posed by background complexity, occlusion, viewpoint, lighting changes, the deformable and articulated nature of hands, etc. Numerous studies have shown that extending the training dataset with real images about synthetic images improves the recognition accuracy. However, little work is devoted to demonstrate what improvements in recognition can be achieved thanks to transferring the style onto synthetically generated images from the real gestures. In this paper, we propose a novel method for Japanese fingerspelling recognition using both real and synthetic images generated on the basis of a 3D hand model. We propose to employ a neural style transfer to include information from real images onto synthetically generated dataset. We demonstrate experimentally that neural style transfer and discriminative layer training applied to training deep neural models allow obtaining considerable gains in the recognition accuracy.

In this paper, we introduce a newly developed target-free calibration method for automotive augmented reality head-up displays (AR-HUDs), which can be performed fully automatically using a smartphone camera. Our method requires no calibration target to be set up in front of the vehicle. Instead, it utilizes feature points of the environment, which makes it robust against misplaced targets and allows for an easy deployment, i.e. in garages. Under the pinhole model assumption, we decouple the perspective projection matrix into three parts: intrinsic matrix, relative pose between the vehicle’s 3D sensor and the smartphone camera, and then rotation between the camera space and the HUD field of view (HUD-FOV). Based on the epipolar constraint, we acquire the relative pose. The determination of intrinsic and rotation matrices is also accomplished without any pre-designed calibration target. The calibration itself takes less than 5 minutes for an eye box with 9 different training viewpoints. With our new approach, we achieve a competitive average reprojection error of 6.7 mm at a distance of 7.5 m, which is comparable to the previous work that applied targets.

In this paper, we propose a novel method for automatic camera calibration based on pedestrians' observations. Our proposed method is capable of estimating calibration parameters for complex scenes having more than one ground plane. Unlike existing methods that require time-consuming optimization step, our method uses real-time re-estimation step based on fuzzy logic while relaxing the assumption on the number of ground planes in the scene. Furthermore, we propose a dominant ground plane detection step for better calibration parameter estimation on complex scenes. To evaluate our proposed method, we run comprehensive testing using 5 different datasets covering varieties of calibration parameters and scene properties, we also conduct tests on a synthetic dataset for more detailed analysis. Test results show that our proposed method outperforms existing state-of-the-art methods in both performance accuracy and time complexity.

Computational modeling of visual attention has been a research field focused on emulating the behavior of biological visual systems in a given scenario, by using mechanisms developed for fixation prediction or salient region detection. In the literature, different approaches have been presented to emulate the interactions that occur in the early vision system of biological structures. However, mathematical modeling of these systems applying theories related to fractional operators could outperform the existing models. In this paper, we present a fractional bio-inspired filter for salient color detection in natural scenarios, based on the behavior and distribution of the cone photoreceptors cells in the retina. The filter was compared with two classic saliency algorithms over a natural color image dataset in terms of saliency prediction and processing time, using a Similarity (SIM) score and runtime performance, respectively. Our approach reach the second best result in therms of saliency prediction with a 48,9% of SIM with ground truth fixations maps and the fastest time response, with an average time of 0.12 s when processing a high resolution image, being 25% faster than Itti et al. algorithm, one of the most applied in robotic vision tasks.

One of the challenges present in the field of tomographic imaging is the reduction of the radiation dose imparted to the object. Monitored reconstruction is one of the approaches to reduce the dose by means of dynamic stopping of the scanning process. In this paper, analysis was performed for the RSRE-based stopping rules for monitored reconstruction, using the previously analyzed RSRE-based reconstruction quality metrics, a widely-used PSNR metric, as well as quality metrics designed to mimic human perception, such as SSIM and ISSIM. It was shown that the unnormalized RSRE-based stopping rule performs better than the baseline for SSIM, ISSIM, and PSNR, and that for the latter the best result was achieved using the stopping rule designed for the reconstruction quality metric normalized to the Radon invariant. The stopping rules were compared with a synthetic a-posteriori “perfect” stopping rule and it was shown that the RSRE-based stopping rules closely approach the perfect stopping rule for the RSRE metric normalized on the Radon invariant, as well as for the ISSIM metric.

ABIDE is the largest open-source autism spectrum disorder database with both fMRI data and full phenotype description. These data were extensively studied based on functional connectivity analysis as well as with deep learning on raw data, with top models accuracy close to 75% for separate scanning sites. Yet there is still a problem of models transferability between different scanning sites within ABIDE. In the current paper, we for the first time perform domain adaptation for brain pathology classification problem on raw neuroimaging data. We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.

We present a method for illuminant estimation that exploits a generative adversarial network architecture to generate a spatially-varying illuminant map. This map is then transformed by consensus into a global illuminant estimation, in the form of a single RGB triplet. To this end, different consensus strategies are designed and compared in this paper. The best solution won second place in the 2nd International Illumination Estimation Challenge, specifically for the indoor track.

The algorithm for 3D vector image reconstruction from a set of spectral tomographic projections collected with CT set-up completed with an optical element or elements inside the optical path behind the sample is proposed. The purpose of their placement into the optical path is to divide the integral polychromatic projection into a series of monochromatic projections, i.e., to get a multi-channel image. Understanding of the reconstruction results in the monochromatic case is beyond question, the relationship between the reconstructed spatial distribution of the linear attenuation coefficient and the discrete description of the elemental structure of the probed object is linear. In difference with monochromatic case the result of the reconstruction from polychromatic projections is a spatial distribution of the so-called effective or average attenuation coefficient, its connection to a discrete description of the elemental structure is nontrivial. However, if the distribution of the averaged coefficient is supplemented by distributions of linear coefficients for several energies, then it is possible to estimate of the local composition of the object. We present a model for the formation of spectral multi-channel projection based on crystal analyzer usage and describe the steps needed to solve the tomography inverse problem.

Recently, adversarial examples become one of the most dangerous risks in deep learning, which affects applications of real world such as robotics, cyber-security and computer vision. In image classification, adversarial attacks showed the ability to fool classifiers with small imperceptible perturbations added to the input. In this paper, we present an efficient defense mechanism, we call DVAE-SR that combine variational autoencoder and super-resolution to eliminate adversarial perturbation from image input before feeding it to the CNN classifier. The DVAE-SR can successfully defend against both white-box and black-box attacks without retraining CNN classifier and it recovers better accuracy than Defense-GAN and Defense-VAE.

The method of Computed Tomography (CT) has progressed throughout the past decade with advances in CT apparatus and program parts that have resulted in an increasing number of CT applications. Today innovative CT Xray detectors have high spatial resolution till a tenth or hundredth of a micron. However, itsfield of view is significantly limited. The object being scanned with a high resolution does not always completely enter in (covered by) the field of view of the detector. The collected projections data may be incomplete. The use of incomplete data in classical reconstruction methods leads to image quality loss. This paper provides a new advanced reconstruction method that demonstrates image quality improvements compared with classical methods when incomplete data collected. The method uses the hypothesis about the consistency of object description in sinogram space and reconstruction space. Input data for the algorithm proposed are incomplete data, and the output data are the reconstructed image and the confidence values for all pixels of the image (reconstruction reliability). A detailed description of the algorithm is presented. Its quality characteristics are based on Shepp-Logan phantom studies.

Recent advances have shown sensor-fusion’s vital role in accurate detection, especially for advanced driver assistance systems. We introduce a novel procedure for depth upsampling and sensor-fusion that together lead to an improved detection performance, compared to state-of-the-art results for detecting cars. Upsampling is generally based on combining data from an image to compensate for the low resolution of a LiDAR (Light Detector and Ranging). This paper, on the other hand, presents a framework to obtain dense depth map solely from a single LiDAR point cloud that makes it possible to use just one deep network for both LiDAR and image modalities. The produced full-depth map is added to the grayscale version of the image to produce a two-channel input for a deep neural network. The simple preprocessing structure is efficiently competent in filing cars’ shapes, which helps the fusion framework to outperforms the state-of-the-art on the KITTI object detection for the Car class.

Advanced Driver Assistance System (ADAS) is a very important part of an up to date vehicle. For achieving highlevel objectives in such ADAS functionality like LKA (lane keeping assistance), LDW (lane departure warning system), FCW (forward collision warning) the quality of the algorithms under the hood must be extremely high. In the last few years, it is common that these algorithms are based on DNNs (deep neural networks) applied to the tasks of semantic and instance segmentation, 2D/3D object detection and visual object tracking. Recent state-of-the-art DNN models as usual solve only one single task from the listed above and running several neural networks is rather computationally expensive and even impossible due to the lack of the GPU memory. One of the approaches used to overcome such a problem is a shared backbone (also called feature extractor or encoder). The backbone consumes most of the computing resources thus the model with a shared backbone achieves better inference performance. Unfortunately, the training procedure for a shared backbone model has several difficulties. The first one is the lack of datasets with all the required and uniform annotation types. The second problem is a more sophisticated backpropagation procedure. In this paper, we consider several methods for multi-task neural network training and present the results of such training procedures on several public datasets with dissimilar annotation types. The shared backbone is applied to the following three tasks performed simultaneously on the road scene: semantic segmentation, 2D object detection and 3D object detection. While the performance of the DNNs with shared backbone increased significantly, we obtained the quality evaluation results, which are quite close to the original separate state-of-the-art DNNs and even outperforms them in some evaluation indices.

In this work we made an attempt to improve 3D detection of pulmonary nodules on CT images using Conditional GANs extended with Adaptive Instance Normalization and combined Wasserstein Loss for data augmentation (DA). Nodule generating GAN model used for DA was built upon an open-source CT-GAN network which provides high and reproducible results in the nodule generation task. For the evaluation purpose we used DeepSEED model, which is a 3D end-to-end one-stage detector. We tested our approach on the LUNA16 dataset, the subset of LIDC-IDRI. The proposed model outperformed the baseline detection model trained on the original dataset by 3% of average sensitivity. The augmentation helped achieve a remarkable classification rate: 91% of sensitivity and 86% of specificity.

Violent action represents a threat to public security, thus intelligent violence detection became one of the important and challenging topics in video surveillance scenarios for this reason there is a growing appeal of video-surveillance systems. Hence, it’s mandatory for the detection of violent or abnormal activities to avert any casualties which could cause any damages. Distinctly, in this paper, it is possible to create a network to learn spatial-temporal information on all subjects of violence rather than going through each concept separately. In order to construct a new concept for violence detection system, we rely on a strategy of a dynamic frame skipping to reduce the complexity of calculation. However, following the regions of interest in the frame, the overall complexity of the calculation is decreased. Withal, the History of Binary Motion Image for n successive images is used for features extraction to facilitate to model the human behaviors. Then, the biggest regions of interest are extracted in order to find the maximum component represented violence action. Finally, deep neural networks involve three stacked Autoencoders and a Softmax are adopted as an exterior layer for classification.

This paper examines optimal look-angles for a camera which is mounted on a small unmanned aerial system (sUAS), that provides for maximized object detection on the ground. Using a generic convolutional neural network (CNN), this research identifies the best angle for detecting a ground target from an aerial perspective. The study involves altering camera angles on an sUAS that is flown along a fixed trajectory and then determining the angle which provides the highest detection rate of predefined objects, which are emplaced at known locations on the ground. The experiment is conducted in simulation and validated on a physical quadcopter. The results of this paper directly influence the U.S. Army’s research efforts on training neural networks and developing object detection algorithms.

Detection of pedestrians on embedded devices, such as those on-board of robots and drones, has many applications including road intersection monitoring, security, crowd monitoring and surveillance, to name a few. However, the problem can be challenging due to continuously-changing camera viewpoint and varying object appearances as well as the need for lightweight algorithms suitable for embedded systems. This paper proposes a robust framework for pedestrian detection in many footages. The framework performs fine and coarse detections on different image regions and exploits temporal and spatial characteristics to attain enhanced accuracy and real time performance on embedded boards. The framework uses the Yolo-v3 object detection [1] as its backbone detector and runs on the Nvidia Jetson TX2 embedded board, however other detectors and/or boards can be used as well. The performance of the framework is demonstrated on two established datasets and its achievement of the second place in CVPR 2019 Embedded Real-Time Inference (ERTI) Challenge.

Shot Boundary Detection (SBD) also known as a temporal video segmentation is a preprocessing task for multiple videos applications, such as indexing and retrieval. The SBD output provides coherent temporal units which are easy to manipulate. The Most previous works implement theirs frameworks based on visual features to measure similarity for transition detection task. However, the video is very enriched by data which could be beneficial. In this paper, referring to recent multimodal works, we propose to introduce the audio components to increase the SBD task. Firstly, we worked on candidate segments obtained by measuring similarity between low features (SURF, HSF) from original video. Then we used deep features obtained from trained model (Resnet-50) for visual similarity and we introduced the audio segmentation based on Power Spectrum Density (PSD) to contribute for transition detection. The proposed method is evaluated on the clip shots dataset. Experiments on this data show that the proposed multimodal approach can achieve a better performance compared with the state-of-the-art of methods that used visual approach.

The paper presents an algorithm for road markings detection in the image. The road markings are approximated by polyline with a restricted maximum curvature angle. To detect a marking segments an image is processed by a sliding window and for each window position, a straight line is detected by calculating Fast Hough Transform (FHT). Further, detected segments are grouped by relative position. Segments groups are then approximated by polylines. The algorithm was tested on real data collected from the front-looking camera of the autonomous vehicle driving at the experimental area “Kalibr” (Moscow). The road marking dataset used to evaluate the algorithm is publicly available at ftp://vis.iitp.ru/road markup dataset/. The precision of road markings detector was evaluated as 43%, and the recall as 73% which is sufficient for the autonomous vehicle precise positioning as demonstrated in [1].