This PDF file contains the front matter associated with SPIE Proceedings Volume 11049, including the Title Page, Copyright information, Table of Contents, Author and Conference Committee lists.

Aquaculture farm provides a solution towards the overfishing phenomena. However, maintaining big scale farms manually requires going through hours of video footages to collect important information about the fishes. These footages are usually taken by an underwater camera affixed in many different ways within the farm’s cage and are manually analyzed by human operators. Since they are limited to biological restrictions, issues such as wandering attention span or human error may occur. This paper proposes a non-intrusive and automated way of extracting meaningful information such as number of fishes from underwater video footages of fishes, using image processing techniques. Experimental results shows that the system managed to achieve 74.59% accuracy for correctly counting fishes.

As a well known filter in the category of order statistic filters, median filter has been widely employed in the fields of image processing, computer graphics and computer vision etc. This paper proposes an approach for implementing the median filter with varying kernel sizes at each different pixels. The proposed algorithm consists of two kinds of histograms. One is column histograms for all columns, the other is a kernel histogram for kernel window. The column histograms are updated according to the scales when moving to the next row, while the kernel histogram is updated using column histograms when moving to the next pixel. Since the kernel sizes are varying at each pixel, the updating of column and kernel histograms uses operations of addition and subtraction depending on the scales. Compared with the brute-force implementation, the experiments show that the proposed algorithm is very fast and effective.

Filtering image by eliminating irrelevant details as could as possible while preserving structure edge and corner becomes very important in the fields such as image processing and computer vision etc. In this paper, we present an Anisotropic Rolling Filter (ARF) for smoothing image while preserving important structure edge and corner. The proposed filter implements an extended cross bilateral filter, in which the range weights are updated in an iterative manner, while the spatial Gaussian weights are computed in anisotropic directions instead of isotropic directions. The anisotropic directions are computed based on structure tensors which are calculated at each pixel to determine structure orientations. Compared with the original rolling guidance filter, it is found that the proposed anisotropic rolling filter has stronger smoothing and structure preserving ability.

This paper presents a classification method of tourism categories based on heterogeneous features considering existence of reliable results. The proposed method performs estimation of existence of reliable results based on one-versus-one scheme from three kinds of classification results obtained from tourism images, geotags and textual tags, separately. Then if the reliable result is included in the above results, this result is regarded as a final result. Otherwise, the final result is obtained by the multiple annotator logistic regression. The proposed method realizes accurate classification by estimating the existence of reliable results from more than two kinds of results.

The documents of the government-general of Taiwan recorded from 1895 to 1945 contain the whole of Japanese official documents before the end of the WW2, and have great historic value. The characters in the documents, however, are illegible because they were written by hand with a brush. It is labor-intensive work for historians or scholars to understand the documents. We propose a method for character recognition of these documents by using a convolutional neural network and also conduct to solve the problem of imbalanced learning data. Experimental results show that the top-1 and the top10 accuracies were 89.48% and 98.10%, respectively.

In the area of electronic noses (e-nose), applications in the field of wine aromas detection are uncommon. The number of qualified human wine experts is low and their cost is high. This paper has been developed for the purpose of recognition of typical aromas in red wines at a low cost. We propose simple linear regression analysis to classify typical aromatic compounds in wine by years of an electronic nose and using feature reduction-based method, principal component analysis (PCA) as feature extraction techniques show datasets of this group of compounds are clearly improved the requirement as follows percent classification rates (performance evaluation). The experiment simple linear regression analysis classification results different types of wine grapes percentage of correlation extract and different years of wine grapes.

Single image super-resolution (SISR) is a classical problem in low-level vision task, which aims to find a mapping function from a single low-resolution (LR) input to corresponding high-resolution (HR) output. Recently, deep networks have achieved great success in SISR task. Recent successful deep models mostly consist of stacked the same size convolution filters whose size is mostly 3 by 3. To cope with more variations in both local dependencies and global contexts between LR and HR images, we propose multi-kernel based deep residual networks for SISR. Since larger kernel requires more parameters, we adopt dilated convolution for increasing the size of the receptive field. Also, we adopt local feature fusion, global feature fusion and local residual learning for controlling the multi-scale features and hence for the better performance by accelerating the convergence. Experimental results show that our proposed model yields improved performance.

In recent years, many SLAM (simultaneous localization and mapping) systems have appeared showing impressive dense scene reconstruction. However, the normal SLAM system build 3D scenes at point level without any semantic information. Many computer vision applications require high ability of scene understanding and point-based SLAM shows insufficiency in these applications. This paper studies about fusing 3D object recognition into SLAM system, using hand-held RGB-D camera and RTAB-Map to reconstruct dense point cloud of 3D indoor scene. Then we use supervoxel based point cloud segmentation approaches to over-segment the scene. 3D object classification model trained by PointNet is added to merge the segmentation process and object recognition. Our experiment on indoor environment shows the effectiveness of this system.

Antique stereographs were taken by photographers more than 100 years ago. In general, the antique stereographs are printed to the about 9cm×17cm thick paper and they are distorted due to aged paper surface. To archiving them, their captured images have to be transformed to the original image. In the paper, a user support system for archiving antique stereographs by distorting input image of a stereograph. In our system, a user selects by clicking multiple points at the edge of the image. The input image is divided into small regions based on the points and each small region is transformed to the same rectangle with the known size by a perspective transformation. To obtain small regions all of which are originally the rectangle with the same size, the points obtained by user’s click are interpolated quadratically. To evaluate the correction accuracy of the proposed system, the experimental results are shown.

We present an efficient resource scheduling scheme for out-of-core dynamic streaming of a 3D scene. The entire scene is stored in the cloud and relevant scene data are streamed to a client mobile device in real time based on a user-selected path in the 3D scene. We analyze the path data in order to yield efficient streaming of 3D urban objects. We compare streaming scheduling based on the user-selected path and on only the user’s current location (i.e. without path lookahead) in terms of number of loaded objects, rendering performance, and storage. The client application is implemented in Unity game engine and we perform experiment on an Android mobile device.

Range image registration is an essential technique for 3D modeling of the real world object from its range images captured by a 3D sensor. It estimates the relative positions and orientations of the viewpoints using multiple range images which are captured by the 3D sensor from multiple viewpoints. The rigid transformations to define the relative positions and orientations of the viewpoints are estimated based on the corresponding 3D points, which observe the same point on the object surface. In this paper, we propose a method for range image registration based on the consistency of the rigid transformation between corresponding point pairs to improve accuracy. We generate triplets of corresponding point pairs and evaluate the consistency because at least three corresponding point pairs are required to estimate a rigid transformation. We estimate the rigid transformation using each triplet of the corresponding point pairs and evaluate the distance between estimated rigid transformations in the parameter space. We select the triplet which have the largest number of consistent triplets and estimate the rigid transformation using the selected triplet and its consistent triplets. In the experiment, our method obtained the most accurate and the most stable registration results compared with the conventional methods, Iterative Closest Point and Globally Optimal ICP.

Recently, image projection attracted a lot of attention, and it is used in various situations. So, the system that realizes image projection simply in any wide areas becomes necessary. Therefore, in this research, we proposed a geometrical correction method for surrounded projection combining a standard lens camera with a fish-eye lens camera and realize simple construction of image projection space in any indoor spaces. By capturing the same pattern images from two types of cameras, two types of cameras are corresponded without external parameters of cameras and projectors, and simplifying obtained data makes the correspondences highly accurate. In this research, we evaluated our geometrical correction method in accuracy, and projected image projection with multiple projectors in the wide indoor space.

A new method of estimating swimmer position in swimming pool video is proposed. The video of swimming games is taken from a higher seat row in audience seat area. It can cover the whole field of a swimming pool. The swimming pool video is transformed so that each lane can be analyzed along with the lane direction. The foreground region that includes both the swimmer and their water splash is extracted by adaptive background modeling and by setting the mask region to cope with the influence of the non-planer water surface. Then, based on the color analysis on water splash, swimmer region can be successfully extracted. The position is estimated as the center of the Gaussian distribution of the swimmer region. The proposed method was applied to a nationwide swimming game.

Image translation is a computer vision task that involves translating one representation of the scene into another. Various approaches have been proposed and achieved highly desirable results. Nevertheless, its accomplishment requires abundant paired training data which are expensive to acquire. Therefore, models for translation are usually trained on a set of paired training data which are carefully and laboriously designed. Our work is focused on learning through automatically generated paired data. We propose a method to generate fake sketches from images using an adversarial network and then pair the images with corresponding fake sketches to form large-scale multi-class paired training data for training a sketch-to-image translation model. Our model is an encoder-decoder architecture where the encoder generates fake sketches from images and the decoder performs sketch-to-image translation. Qualitative results show that the encoder can be used for generating large-scale multi-class paired data under low supervision. Our current dataset now contains 61255 image and (fake) sketch pairs from 256 different categories. These figures can be greatly increased in the future thanks to our weak reliance on manually labelled data.

The main contribution of this work is developing an end-to-end air-writing recognition technique for a real-time application. We assume the user performs the air-writing naturally and intuitively without doing any explicit signal. For avoiding the spotting process, this work considers the segmentation free technique using the LSTM network with CTC loss. The fusion scheme models the writing trajectory with the spatial and temporal features. To extract the writing information from the finger motion, we utilize a window-based technique for segmenting stream data for generating the training features. We deploy two features: the hand position and the path signature, to train the proposed network. For evaluating the performance of the proposed technique, we conduct the experiments the public dataset namely the finger writing. From the result, it confirms the fusion scheme can improve the recognition accuracy. The appropriate size of the sliding window for the proposed structure is 0.25 second while the skip size equals 83 milliseconds. The proposed network can recognize the air-writing word 75.81% without the language model. When considering the processing time of the recognition technique, the air-writing could predict the written word within 6.37 milliseconds. It confirms the proposed algorithm can deploy for a real-time application.

Large displays are capable of visualising a large amount of data on multiple views including scatterplots and parallel coordinates and are often present in meeting rooms. They can be used to interact with a dataset and foster discussion among team members. Although some of these large screens have multi-touch capabilities, in many cases it is cumbersome to have to stand close to the display in order to interact with it. One of the solutions is to use a small handheld display to interact with the large display. This paper discusses how traditional interactions such as selection, brushing, and linking can be performed using a secondary handheld device. As a proof of concept, a system including scatterplots and parallel coordinates views is implemented. The interactions are straightforward and are useful for any interactive visual analysis application on a large display with wireless connectivity.

In the medical field, diagnosis based on images/videos is constantly becoming increasingly important. Recently, an 8K resolution endoscope has been developed. Lighting is necessary to obtain sufficiently lighted images/videos when endoscopic examinations or operations are performed. Although it is necessary to have sufficient illumination, the lighting condition of endoscopes is limited owing to the heat generated by the light source. Poor lighting conditions always produce noise. Particularly, the size of an 8K imaging cell is small, and the number of photons for it is limited. Noise is an important issue in 8K endoscopic images. During a diagnosis, in order to obtain detailed information on an ailing part, same areas are often monitored repeatedly. 8K endoscopic videos have a high resolution; having a high image quality in each frame of still images is important used during diagnoses. However, unlike videos, in still images, observers can easily distinguish not only resolution differences but also degradations, such as noise, in problematic areas. In this study, a total variation (TV) denoising method is used to reduce the noise of 8K endoscopic images.

Due to the advances of imaging devices technologies, high quality video has become easy to produce. There are still problems to be solved though. Gaussian noise is always mixed in images by photo-electric conversion. Noise reducer (NR) is a signal processing method to cope with the issue. However, noise level is required for NR to work effectively. Since signal processing methods for video should work in real-times, the noise level also should be detected in real-times as well. In this paper we propose a fast and accurate noise level detection algorithm.

The damage caused by the pirated films amounts to $1.3 billion a year. The pirated films are mainly created by re-shooting a screen in a theater or duplicating the official DVDs. Films are released in a theater. Before the official DVDs are released, 90% of the films are pirated in a theater and illegally made DVDs are sold. Although counter technologies are required, there is no effective proposals. The important requirement for the technology is that it does not degrade the image quality in a theater. It only degrades the re-shot video. In this paper, we report a novel method to make it possible.

In recent Japan, flood disaster is frequently caused by typhoons and torrential rain. However, people’s awareness of flood disaster is not high, and low evacuation rates are in question. Since existing research and contents only display the water depth at flooding, it is difficult to fully tell the danger of flood damage. In this research, we aim to develop a system that can tell the risk of flood disaster through the experience of walking on a flooded road using virtual reality (VR).

Fiber-wireless (FiWi) access networks are typical hybrid access technologies that combine the high bandwidth of optical access and the flexibility and ubiquitous coverage of wireless access. Mobile edge computing (MEC) provides low latency services and cloud computing capabilities at the edge networks. The convergence of MEC and FiWi access networks can improve network performance and QoS (Quality of Service) of augmented reality (AR) services. Therefore, in this paper, we study the problem of integrating MEC with FiWi to enhance AR services. We first present the integration scheme of MEC with FiWi for AR applications. We then propose our AR applications enhanced by the integration of MEC and FiWi access, which can provide multiple low-latency AR services. Performance evaluation results demonstrate that the integration of MEC with FiWi can support long-distance AR applications with low delay.

Most Virtual Reality (VR) headsets come with cable connections to their computing units. This is inconvenient for deployment in immersive interactive environments. A wireless VR solution is a promising approach for improving the user experience. The Mobile Edge Computing (MEC) provides high bandwidth and low latency, which enhances VR wireless solution for supporting high definition and low latency applications. In this paper, we propose a VR wireless solution based on MEC to meet the requirement of future VR applications with high resolution display and low latency. Compared to existing wireless VR solutions, our solution is superior in improving the video quality and enhancing the quality of the user's experience.

In this study we developed a visualization system of human motion characteristics using motion data. Our system visualizes user-defined characteristics with selected visualized methods using 3DCG. For kinematic analysis, the system can project human motion to frontal plane, sagittal plane, and horizontal plane. In this study, we analyzed one classic ballet movement called arabesque obtained by a female dancer to compare over time. We confirmed that the angle of hip adduction and the lateral tilt of the pelvis increase with her training. In addition, we confirmed that the dancer kept standing on tiptoe while leaning the core of the body backward.

Human detection is a technology that is used in various fields. It is, however, a trade-off between detection accuracy and precise extraction of human regions. The purpose of this paper is, therefore, to extract a precise human region using Coarse-to-Fine Method or Human Skeleton Method. In the Coarse-to-Fine Method, at first, Coarse Detector detects multi-posture humans. Next, Fine Detector extracts precise human regions. In the Human Skeleton Method, human skeletons are extracted by using OpenPose. Next, skeleton images are dilated based on physique of the human. Finally, human regions are extracted by using GrabCut. The extracted results are evaluated by detection accuracy and preciseness of extracted regions. F-measure and IoU (Intersection over Union) are employed to evaluate the detection accuracy and the preciseness respectively. In the Coarse-to-Fine Method, the detection accuracy is 0.523, and the extraction preciseness becomes 0.807. In the Human Skeleton Method, the detection accuracy reaches 0.928, and the extraction preciseness is 0.868. Especially, Human Skeleton Method gets excellent performance in not only the extraction preciseness but also the detection accuracy.

This paper presents gaze-based visual feature extraction via Discriminative Locality Preserving Canonical Correlation Analysis (DLPCCA) for visual sentiment estimation. The proposed method calculates novel visual features reflecting users’ visual sentiment by applying DLPCCA to gaze and original visual features. Consequently, accurate visual sentiment estimation becomes feasible by utilizing the novel visual features derived by the proposed method.

Manufacturing technology for counterfeit brand items has advanced in recent years, and it is becoming very difficult for humans to identify counterfeit products. In this paper, we propose an inspection system using image matching methods to realize the authenticity inspection of brand logos through image recognition. In the first experiment, we compare the similarity evaluation performance by NCC (Normalized Cross-Correlation) and POC (Phase-Only Correlation) using images of actual brand products. In the next experiment, we confirm the effectiveness of logo region detection processing using edge images as preprocessing of image matching with the aim of improving inspection accuracy of images containing many background components. Experimental results show that it is possible to separate genuine and fake more accurately by evaluating similarity by POC. Moreover, we confirmed that by adding the logo region detection processing, the background component of the image was reduced and highly accurate inspection was possible.

In mobile robots and autonomous vehicles, the information of depth and camera’s orientation are necessary for the applications of positioning and navigation. In this paper, we propose an appearance-assumption-based method for the detection of horizontal line (HL) or vanishing line (VL) on the image observed with a single omnidirectional camera. HL can provide the important information about depth and camera’s orientation for the single view estimation. The detection of HL is assumed as an optimization problem. The optimization of the line is adopted by the genetic algorithm (GA). We implement the real-time detection method of HL using GA. Therefore, the proposed method is evaluated on the multiple videos with the various environment. Experimental results demonstrate the efficiency of the proposed method.

The main purpose of computer games is to immerse a user in a virtual environment. The perception of immersion experience has been used to develop virtual reality applications for game-based learning, healthcare, rehabilitation, etc. In traditional games, the user immersion is performed through the sense of visuals and sounds with interactions through 2D input devices like the mouse and keyboards. Haptic technology can simulate tactile and kinesthetic sensations in virtual environments. The real-time interaction and feedback are very important for development of rehabilitation applications. The main goal of our research is to enable real time simulation of force feedback for rehabilitation and learning games. This paper presents a framework for developing the virtual puzzle-box game for rehabilitation and education based on haptic display. The proposed framework introduces a collision handling method to compute feedback force and provide realistic feedback through the haptic device.

In this paper, we aimed at discrimination of defects under conditions where there is a large number of good products and a small number of defective products. Although automation of a visual inspection is essential to improve the quality of products, either or both of the features extracted by the experts and balanced dataset are needed. We tackled such a problem. By combining AAE, which can extract features following any distribution and Hotelling's T-Square, which is an effective anomaly detection method when data follows a normal distribution, it is possible to discriminate defects with a small number of defective samples.

Writer identification is one of the active areas of research. It is important to prepare a large number of characters of the same class to improve the accuracy of writer identification. However, it is not always possible to prepare enough characters of the same class. In this case, handwriting examiners compare different classes of characters and analyze using common handwriting parts for each character. However, this is very difficult. Therefore, we assume that handwriting characters written by the same writer have features independent of character classes. In this paper, we propose methods to extract features that are independent of character classes using deep neural networks. We used Conditional Variational AutoEncoder (CVAE) as a learning method. A writer identification experiment shows that these methods can extract independent features of character classes, and extracted features are useful in writer identification. Furthermore, we examined the relationship between human interpretation of character features and accuracy of writer identification by using character features extracted by disentangled feature extraction methods.

We propose a method to restore the original condition of the cultural properties in the photographed image using extraction and texture synthesis of the degraded parts. It is judged whether or not degradation such as a crack has appeared in each block divided from the image by machine learning. The cracks in the selected block are extracted by binarizing the pixel values and restored with the texture synthesis technique. We confirmed by our computer simulation experiments that the proposed method showed good results.

This paper describes one of the assistance methods for annotation tasks of sign language words using binary action segmentation. The binary action segmentation divides a sign video into binary units, which correspond to during sign and static posture. At this time, the user's annotation tasks can be reduced from the full-manual work to inputting labels and correction of the segmented units. The proposed binary action segmentation is composed of Support Vector Machine and Graphcuts. The trained Support Vector Machine classifies each frame into Motion or Pause, and Graphcuts refines the initial segmentation. We evaluated the proposed method with a Japanese sign language words database. The database includes 92 Japanese sign language words which are signed by ten native signers. The total number of videos is 4,590, and 3,800 videos of 76 words except for recording and sign errors are used for the evaluation. The proposed method achieves comparable result with a smaller amount of training data than the previous method. Moreover, the work reduction ratios of annotation tasks using an annotation interface were 26:17%, 26:34%, and 17:88% for the sets whose the numbers of segmented units were 2, 3, and 4, respectively.

This paper presents a novel estimation method of field positions in soccer videos using Convolutional Neural Network (CNN)-based image features. CNN-based features have been well known to be effective for tasks in machine learning. Therefore, the proposed method adopts CNN-based image features. By using these image features, the proposed method enables accurate estimation of soccer field positions than handcrafted features, i.e., Hough transform-based features. We show the effectiveness of our method via experiment results using actual soccer videos.

We propose a fast image dehazing algorithm based on multi-scale guided filtering. We first construct an image pyramid from a hazy input image. Then, based on the observation that the most time-consuming procedure is the estimation of the transmission map and atmospheric light, we estimate them only at the coarsest level. Then, we obtain the transmission map at the finest level by upsampling and guided image filtering. Experimental results show that the proposed algorithm is capable of real-time operation, while providing comparable or even better performance than the state-of-the-art algorithms.

In this research, in order for autostereoscopic display of dual projection types to display the image with larger parallax, we improved a new optimization method of the two projector images. We achieved that aim by deriving a new update formula in addition system and adjusting the target light field and weight matrix to create the area that saturates light.

In this work, we propose a static video summarization approach for the analysis of cattle's movement. An original digital video with a length of about 1 hour (58 minutes and 42 seconds) recording the daily behaviors of cattle in cattle barn was applied as the experimental object. In the approach, machine learning, statistics and color histogram are used to extract key-frames from the video. And it can be confirmed that this approach achieve the purpose of video summarization based on cattle's movement through analyzing the results of the experiment.

We have developed a video analysis method of shoot motion in basketball formation of practice. Our method can estimate accurately the position and movement of players from practice videos, and judge whether the player's shoot has succeeded. And by combining these, it is possible to estimate the shoot timing of players. Finally, it is possible to calculate the shoot probability by each positions. We evaluated the proposed method and also verified its usefulness. By this method, basketball formation practice and shoot practice can be performed efficiently at the same time.

In recent years, in order to display hi-vision broadcast the next generation displays, super resolution techniques for improving image resolution are demanded. In addition, with the spread of digital cameras and smartphones, people have more opportunities to handle camera images. In particular, images of surveillance cameras are required to obtain highdefinition output by removing noise. In this paper, in order to avoid the adverse effect of image quality deterioration when emphasizing noise mixed image which is a problem of super resolution processing, we examine a noise removal method before super resolution processing. In our proposed method, Total Variation regularization, which is decomposed into structure and texture components, is extended in direction of time axis. As a result, moving images can be decomposed into structure moving images and texture moving images. In theory, it is thought that noise components with large value of Total Variation is shift to texture components. Furthermore, we aim for separation of texture components and noise, and aim for acquisition of high-definition moving images. We verify the performance of our proposed method by comparing it with the BM3D method, which is regarded as the highest performance for moving image noise removal processing.

The unreconstructed historic sites are hard to attract human interests or function as sightseeing spots. As a means of virtual reconstruction and appreciation of historical sites, the location-based Augmented Reality was used to develop applications for smartphones. Main development comprises 3D model display using position information from GPS and virtual space update according to user's movement. Since this application is assumed to be used for sightseeing purposes, additional functions are required to be implemented for making it more convenient.

It is difficult to present natural avatar's facial expression. We propose a new method for generating avatar's natural facial expression by combining (1) user’s emotion classification based on his/her facial images analysis, (2) the degree of emotion based by biometric heart rate sensor. Generated avatar’s facial expressions are enhanced by referring emotion classification results and the degree of emotions measured by biometric sensor. Finally, “Manpu” (comic symbols) is adding to visually express the user’s emotions, feelings or sensations.

We propose a real-time rendering method of Japanese ink painting. Especially, we focus on methods both "mokkotsu" and "horinuri" techniques. We developed a real-time rendering system which can render Japanese ink painting images based on two techniques.

In this paper, we study dense multi-view systems transmitting light fields beyond a visual obstruction as if it were transparent. Multi-view 3d displays appropriately combined with camera arrays or lens arrays provide consistently augmented reality for many users at the same time as a simple solution of such occlusion in the real world. We first describe a design of our proposed multi-view system, appearances of which are evaluated by assuming state-of-the-art visual devices to be used. Then, in order to reduce image sensors acquiring light fields for inexpensive systems, efficient interpolation is introduced to reconstruct the entire 4d light field in real time for consistency with the real world. We show that our practical implementation on a GPGPU actually achieves real-time interpolation obtaining enormous data of light fields without severe degradation for various depths.

This article proposes a self-training system of chest compression, one of the important methods of emergency life-saving, with augmented reality (AR) technology, which decreases barrier of the practical learning and contributes to improvement of the learner’s skill. The system is built on a see-through head-mounted display (HMD) with camera and computer. For a user wearing the system who practices the chest compression with a training manikin by him/herself, it provides feedback on posture, depth and rate superimposed on the HMD. Making the user easily learn the chest compression by his/herself in real-time without his/her additional actions/movements such as confirmation of his/her operation by video, the system is expected to contribute to a practical and fundamental self-learning tool of the emergency life-saving operation.

For beginners of sign language, this article proposes the arm motion learning support system to analyze the difference between the example of teacher's motion and user's motion, and to urge the user to improve his/her motion. Using virtual reality (VR) system composed by head-mounted display and motion tracking hardware/software, the proposed system measures how similar between the example motion and user's motion, and gives him/her the result of analysis, followed by the motion replay as visual feedback. For the user's easy understanding, the system scales the character according to the user's physique. Also the system adopts the sight cursor input for hands-free input. Questionary investigation after user test by 21 people shows that the proposed method has the competitive advantage against the conventional learning methods (textbooks and videos) especially in interactivity including visual feedback, depth sense in posing, motivation in learning, and easy user-interface. Detection of fingers' motion should be discussed in the next step. Since heavy and/or bulky devices are unacceptable for the purpose of the self-learning, a suitable device or other methods will be surveyed.

In recent years, three-dimensional technology is capable to simplify the reassembly process of the fractured objects. The reassembly process of stone tools can be easily done by the refitted flake matching process. The refitted flake matching process is required for highly accurate segmented flakes. This work aims to extract highly accurate flake surfaces and to match the extracted flake surfaces. In addition, this paper studies the evaluation of the matching accuracy of the extracted flakes and comparison to other methods. This paper also presents an algorithm of flake surface recognition based on feature lines of stone tools. The implementation of this work can recognize flake surfaces according to the scale drawing of the stone tool with high accuracy.

We have been researching and developing virtual museums that can enable a user to appreciate artworks with much reality and less stress as if one is experiencing in actual museums. For this time, as a method to further increase the reality of the virtual museum, we developed a virtual museum using Head Mounted Display. We used Oculus Rift and Unity Game Engine to develop this VR museum. We also performed a comparison of the developed VR museum and the screen-based virtual museum in order to maximize the user experience of the virtual museum. Four aspects of "Freedom of operation", "Immersion", "Comfort of play", and "Picture quality" are examined to clarify which type of virtual museum would be suitable for any specific user's needs.

Recently, along with rapid development of the image processing technology, image processing has been adopted in various fields for various purposes. Development of an intelligent machine that mounts a camera as an eye is a thriving technology, and it is employed not only in industrial fields but also in the fields involving ordinary citizens. Especially, development of Intelligent Transportation Systems is very active and many methods of detecting human and automobiles have been proposed using laser radars, LIDARs or in-vehicle cameras. However, they remain only on the detection of the presence of such objects and the methods to detect rush-out objects into a road have not been developed yet. In this paper, a method is proposed which detects a human from an image with his/her body direction information. This intends to detect a human who might rush out into a road in front of an ego-car. In order that the human model used for extracting the feature may capture the appearance of human rush-out properly, directional human models and classifiers are introduced. The proposed method was examined its performance experimentally and the effectiveness of the method was shown/ satisfactory results were obtained.

With the advent of deep convolutional neural networks (DCNN), the improvements in visual saliency prediction research are impressive. Despite this, it is still needed to fully characterize the multi-scale saliency-influential factors into the current deep saliency framework for further improvement. However, the existing approaches aiming at capturing multi-scale contextual features either suffer from the heavy computation or limited performance gain. To overcome this, a lightweight yet powerful module for fully exploiting multi-scale contextual features is desired. In this paper, we propose a DCNN-based visual saliency prediction model to approach this goal. Our model is inspired by the GoogleNet, which use the inception module to capture multi-scale contextual features at various receptive fields. Specifically, we revise the original inception module to have more powerful multi-scale feature extraction capacity and less computation load by utilizing dilated convolutions to replace the original standard ones. The whole model is trained end-to-end and is efficient to achieve real-time performance. Experimental results on several challenging saliency benchmark datasets, including SALICON, MIT1003, and MIT300, demonstrate that our proposed saliency model can achieve state-of-the-art performance with competitive inference time.

With the growing number of population in the world nowadays, novel human-computer interaction systems and techniques can be used to help improve their quality of life. A gesture based technology can help to maintain the safety and needs of the disable as well as the general people. Gesture recognition from video streams is a challenging task due to the high changeability in the features of each gesture with respect to different person. In this work, we propose a vision-based hand gesture recognition from RGB video data using SVM. Gesture-based interfaces are more natural, spontaneous, and straightforward. Previous works attempted to recognize hand gesture for different scenarios. Throughout our studies, gesture recognition system can be based on wearable sensor or it can be vision based. Our proposed method is applied on a vision based gesture recognition system. In our proposed system image acquisition starts from RGB videos capture using Kinect sensor. We convert the image frames from videos to blur for background noise removal. Then, we convert the images into hsv color mode. After that, we do the dilation, erosion, filtering, and thresholding the image for converting to black and white format. Finally, using the prominent classification algorithm SVM, hand gestures has been recognized. In conclusion, the framework aims to create a better vision-based hand gesture recognition system with novel techniques.

Maximum-likelihood expectation-maximization (ML-EM) method and multiplicative algebraic reconstruction technique (MART), which are well-known iterative image reconstruction algorithms, produce relatively highquality performance but each of which has an advantage and disadvantage. In this paper, in order to compensate for both disadvantages, we present a novel iterative algorithm constructed by a nonautonomous iterative system derived from the minimization of an α-skew Kullback–Leibler divergence, which is considered as a combined objective function for ML-EM and MART. We confirmed effectiveness of the proposed hybrid method through numerical experiments.

In this study, we applied a novel system “VISTouch” to visualize medical imaging information. We enhanced the system to enable multiple devices to be used in combination dynamically and spatially. When a mobile device (parent) is placed horizontally, it displays an image that can be viewed from above. When another mobile device (child) is placed on this display, relative positions between them are dynamically acquired. The child device can then displays images viewed from its contact position, direction, and angle in real time. By using this VISTouch system, we achieved the visualization of medical images from arbitrary cross sections and arbitrary viewpoints in real time. It was also possible to indirectly view different cross sections as well as a general overview. Moreover, we attempted to visualize information from various angles on the actual scale of the human body using the installation artwork “CUBISTA,” which was built using VISTouch Air technology. From these implementations, we confirmed the potential of VISTouch technology to visualize medical imaging information.

The goal of our research is to describe the growth of a tumor region captured in a temporal series of MRI images. The tumor growth description requires registration of the given images in order to compensate the body deformation. It should be noted that one can apply an existing non-rigid registration technique for the compensation but it would deform the tumor region in a source image and the resultant description of the tumor growth would be inaccurate. We hence developed a Large Deformation Diffeomorphic Mapping (LDDMM) method that can non-rigidly registers given images while keeping the shape of the tumor region in a source image. The proposed method was applied to a series of MRI images of a KPC mouse and the results showed our method could successfully described the tumor growth.

Recently, curvature filter (CF) has been developed to implicitly minimize curvature for image processing problems such as smoothing and denoising. In this paper, we propose a parallel curvature filter (PCF) that performs on GPU which is much faster than the original CF on CPU. Inspired by Convolution Neural Networks processed by GPU, the convolution operations in curvature filter computation can be similarly paralleled by GPU so that the PCF on a single GPU can process 33.2 Giga pixels per second. Such performance allows it to work in the real-time applications such as video processing and biomedical image processing, where high performance is required. Our experiments confirm the efficiency and effectiveness of the PCF.

In traditional stereo matching, global approach is more accurate but time consuming, also have high accuracy in occlusion area. On the contrary, local approach is usually fast but have bad performance, and easily influenced by noise. This paper proposed a novel method to compute disparity between two images. It is based on local approach, but our new cost function aggregated the cost in global way. This aggregation is processed by a weight map which created by the bilateral filter. Every pixel transfers its own cost information to all pixels on the same object, but this information would be restricted by the weight map. After finishing preliminary depth map, we use L-R check to find occlusion and mismatch pixels to refined our depth map. These refinement mechanics fix occlusion areas by the smallest disparity nearby. At last, we use bilateral filter clean up whole depth map. All of above computing process can be parallelized on GPU or cloud sever. Although this algorithm is designed for lowlevel machine, it still exerts high performance in high-level hardware.

In human activity classification, detecting speaking activity can be applied further in behavior analysis such as student learning behavior in an active learning environment. This paper presents a method for classifying whether or not a person is speaking based on lib movement in a video sequence. Assuming that a person of interest is tracked within a room using multiple cameras, at least one camera can capture the face of a target person at every instant of time. Using this sequence of frames of a target person, this paper proposes a method for continuously deciding whether the person is speaking. Firstly, head part is segmented based on (1) the head's top position, (2) head's width and golden ratio of head's height and width. Secondly, the face area is extracted using a skin detection technique. Thirdly, the mouth area in each frame is segmented based on its geometry on a face and a mouth has different color from face skin. Next, mouth opening is roughly detected based on the fact that the opening area has a darker gray level than its average. Finally, only frequency components between 1 Hz to 10 Hz of the detected feature signal is extracted and used to classify the speaking activity by comparing with a threshold. The proposed method is tested with 3 sets of videos. The results showed that the speaking classification and mouth detection achieved 93 % and 94 % accuracy, respectively.

When the printed material is imaged by a monocular digital camera, geometric distortions caused due to folds result in a different appearance from the content of the original printed material. This study aims to reproduce appearance by correction the obtained image. As a proposed method, the geometric distortion is corrected by deforming each local area after dividing the printed material image into local areas. In addition, the brightness change by shading is also corrected.

Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80% of e-book market. In recent years, metadata extraction from manga image has been studied for providing e-comic service. Manga character is one of the important contents for story understanding. In conventional research, some character identification methods are proposed those classify characters’ face images using k-means clustering. However, there are two problems. First, kmeans method needs to specify the number of clusters, however the number of characters in target manga images is commonly unknown. Second, manga includes characters with few appearing, so it is difficult to classify characters with high purity. To solve these problems, we propose clustering method using DBSCAN which decides number of clusters automatically and is robust to noise data. In our prior research, it is experimented that character face clustering using DBSCAN and general CNN features. However, general CNN model is difficult to capture detailed features of manga characters. In this paper, we apply DBSCAN to fine-tuned CNN with manga character faces to improve the clustering accuracy. We also compare the optimal parameter determination method of DBSCAN. Experimental results showed that the dimensional reduction using Kernel PCA and UMAP is effective. In addition, we confirmed the validity of proposed method that determining the parameters of DBSCAN based on the slope changing of k-distance graph.

In an active learning environment, a student activities is crucial to his/her learning achievment. However, keeping track of the student activities by teaching staffs is almost impossible. Hence, using technology for such tedious but important job has become attractive or even necessary. Focusing on such environment, this paper proposes a method of classifying whether a student is writing or reading or working on other things such as doing experiments based on sequential image frames from a single camera. For each frame, an area including the student is cropped out using a background subtraction and thresholding. Then, using the skin detection technique, face and hands of the target students are detected. Such face and hand areas of n sequence of frames are combined as a Gait Energy Image (GEI), which is being used as feature images for the classification in which the Principal Component Analysis is applied. A sum score of the PCA in which each row as an observed sample is taken as a feature while another score of PCA in which a column is considered to be an observed sample is taken as another feature. Using the support vector machine, the two features are used to classify whether a student is “reading” or “not reading” first. Then, for a “not reading” sample, it is classified whether it is “writing” or “doing experiment”. Based-on a sequence of simulated activities, the proposed method can classfy between “reading” and “not reading” with 93% accuracy while the classifying between “writing” and “doing experiment” class achieved 90% accuracy.

In this research, we propose a novel distortion-resistant visual odometry technique using a spherical camera, in order to provide localization for a UAV-based, bridge inspection support system. We take into account the distortion of the pixels during the calculation of the 2-frame essential matrix via feature-point correspondences. Then, we triangulate 3D points and use them for 3D registration of further frames in the sequence via a modified spherical error function. Via experiments conducted on a real bridge pillar, we demonstrate that the proposed approach greatly increases the accuracy of localization, resulting in an 8.6 times lower localization error.

Identifying the potential factors that contribute to an aircraft engine’s durability is a big challenge. In this paper, we propose an interactive system to link the engine maintenance history data, flight route history data, and volcanic ash eruption data, and then visualize the relations among them. Several kinds of visualization forms are used to present the datasets. We implement the React-Redux framework to manage the visual components in the frontend, along with a backend to handle the database. This highly tailored system shows how datasets of different sources can be linked and visualized with interactivity. Data analysts and developers can benefit from full-stack development process from data to intuition, meeting the requirements of customized visualization.

We have already proposed a method to simulate and express the process to rust the iron based on a voxel-based 3D automaton method. However, this method is too slow to simulate the process of rusting the iron. Thus, the previous system can simulate the rusting process of only small objects. In this paper, we propose a fast simulation method of the process of rusting the iron, so that we can deal with large and complex 3D models. As a result, our system enables to simulate the rusting process of iron polygon 3D models over time with high speed.

In the VR content, one of the main reasons for damaging immersive experience is VR sickness. It is well known that exercising user’s arms or legs is effective to control symptoms of VR sickness. In this paper, we propose a new method that wraps the sensor around the wrists and allows natural arm swinging. We also evaluate how to swing users’ arms and walk around virtual space from its walking speed, routes and different types of sensors.

Expression related to blood such as bloodshed and splashes is often treated in the CG field such as games and movie. Along with the improvement of technology and equipment performance, expressions such as blood adhering to objects became realistic and high speed, but on the other hand state change due to time course such as coagulation and discoloration has not been expressed yet. We proposed a method to expand the SPH method, which is a fluid simulation method, to simulate changes in the state of blood over time and aim for more realistic expression of blood. However, our method has a problem of increasing in calculation load. In this research, we accelerated process by voxel division of simulation space and GPU computing.

This paper proposes a flake surface segmentation method using ridge lines extracted from point clouds. First, ridge lines are extracted from a point set of a stone tool. Next, neighboring points of a ridge line are extracted by calculating the Euclidean distance from the ridge line. An area surrounded by the neighboring points of the ridge line is selected. Then, the area is expanded from the selected region to the ridge line to segment a flake surface.

Currently, Virtual Reality (VR) content production is actively performed. While, VR sickness has become a problem. As a means to reduce VR sickness, bodily sensation-type game contents [1] are attracting attention. However, interfaces proposed in the previous work were not sufficient enough for operability. In this paper, we propose “Surf Disc” as an interface for moving around the virtual space freely and smoothly. For university students, evaluation experiments were conducted focusing on VR sickness, body fatigue, and operability by Surf Disc. We conducted experiments to evaluate the operability of “Surf Disc” from the three viewpoints of VR sickness, physical fatigue, and easy-to-use. We verify “Surf Disc” is sufficient and intuitive to move around the virtual space.

Recently, luminance compensation that realizes image projection canceling the in uence of patterned projection surfaces, has attract attention. This technique can cancel the in uence of the patterns, based on the response function representing the input-output relationship between a projector and a camera. On the other hand, it largely depends on the inter-pixel correspondence between the projector and the camera, so projection surfaces is limited to a rigid body. That is, the utilization area of luminance compensation is largely limited. In this research, we realize luminance compensation on swinging curtains by estimating the inter-pixel correspondence in real-time. In addition, the GPU is used to execute the system in real-time. We examine the effectiveness of the constructed system, using curtains of various materials.

In conventional design CAD system, designing the intended curve segment is time-consuming and labor-intensive due to the nature of the curves. Though there are tools within CAD to help reconstruct curves, their performance has been reported insufficient. In this paper, we present a method for curve reconstruction that is based on The Experiential Design Guidelines to facilitate design work. Firstly, the input curve is subdivided according to the guidelines, then reconstructed by the curve interpolation technique. Finally, the resulting curve is optimized to fully meet the Experiential Design Guidelines. The objective function to be minimized is a weighed combination of two components associated with the coherence to the original design, and the curvature approximation by second derivative. The importance of the two components is weigh by a user-defined parameter. Numerical examples demonstrated the effectiveness of the method. However, there remains some problems for further improvements.

Magnetic Resonance Imaging (MRI) was used for analyzing speech articulations. A synchronized sampling method, which is that a subject person repeats speaking a same pseudo word at a tempo by hearing click sounds, was used to enable capturing speech articulators' motion data from relatively slow imaging. For Japanese syllable sequences of /a-ei- u-e-o-a-o/, /ka-ke-ki-ku-ke-ko-ka-ko/, and /ga-ge-gi-gu-ge-go-ga-go/, positions and motions of speech articulators were compared for analyzing detailed differences of speech articulations depending on patterns of consonants and vowels. To analyze time series of narrowing positions of a vocal tract caused mainly by tongue motions, a kymograph method from MRI images is proposed. A thin slice is taken from MRI image in a vertical direction relative to a vocal tract. This thin sliced image has upper and lower boundaries of a vocal tract, that is, a hard or soft palate and an upper surface of a tongue. The same slices are taken from all frames and then they are put in a line in a sequence of time. This sequence of sliced image is looked like a kymograph. It will be a useful tool to observe time series of distance changes of one section of a vocal tract, caused by tongue motions.

Performance of digital radiography detectors can be evaluated from measuring the noise power spectrum (NPS), modulation transfer function, and detective quantum efficiency (DQE). Hence, measuring these quantities with high precisions is important. In this paper, to measure the quantities, we introduce a program, which is called DRDQE (digital radiography DQE). DRDQE supports the IEC62220 standards for various radiography detectors. Furthermore, DRDQE can measure NPS and DQE with high precisions based on Small-Subimage Algorithm. Compensated Partial-Sum Algorithm is also adopted to measure a high-precision NPS at zero frequency. We briefly introduce the employed algorithms in DRDQE and then show an example of DRDQE usage for a direct flat-panel mammography detector. The high-precision measurements from DRDQE were compared with the IEC62220 standard.

In order to observe the noise property of the radiography image detector, the noise power spectrum (NPS) is usually measured. NPS curve can be measured from the average of the periodogram samples. Here, we can improve the measurement accuracy and precision by increasing the number of periodogram samples and spectral resolution. In this paper, we observe the accuracy and precision of the NPS measurement, which is based on the periodograms. For real digital X-ray images, which were acquired from an indirect CsI(Tl)-scintillator radiography detector, we observed the accuracy and precision of the NPS estimates for various resolutions and sample sizes. For a given image size, we could find an appropriate combination of the sample size and spectral resolution from minimizing the mean square error.

ICH is a type of stroke caused by internal bleed of the brain. Accurate and fast measurement of the blood clot volume is important for neuro surgeon to make an informed decision if the patient need surgery or not. In this paper, we present a mobile application “ICH 3D”, which is able to measure blood clot volume on-site using a smart phone with improved accuracy and within a minute.

We have been studying and developing the real-time Computer Graphics (CG) based virtual museum where a user can walk through to appreciate artworks digitized in high-resolution. Our virtual museum also has a function to automatically create TV program-like CG animations using 3D CG models in the virtual space as it is so that the user can learn about individual works by watching the art shows. The CG animation is produced with TVML (TV program Making Language) engine implemented on the virtual museum. However, the current problem is that it requires a lot of work for a developer to write the complicated TVML scripts manually. Therefore, this time we have developed a special tool to help the developer to prepare the TVML scripts easily. With this tool, the developer can produce the TVML-based art program simply by writing out a simple scenario on an ordinary text editor. In order to design this tool, TV art programs actually broadcasted are analyzed to determine the syntax of the simple scenario. Based on the analysis, we have developed the tool with TVML engine working on the Unity game Engine. We have also used this tool to imitate the broadcasted TV program to validate its usability.

We study how music can be played with a computer by tracking hands in the air. We begin our considerations by analyzing how the theremin can be played—an electronic musical instrument which is played without physical contact by the hands of the performer. We then look at the present hand-tracking technologies available for common personal computers and mobile devices and hypothesize that optical tracking devices, like Leap Motion controller, may be used for simulating the theremin functions with a common personal computer. We then describe our own implementation of the digital theremin and analyze why the theremin is considered as the most difficult to play musical instrument and how its deficiencies can be overcome with the help of computer graphics.

Scratch art is an artwork created by scratching on the black paint. After scratching, a part of the colorful background is emerging appear from the scratched region. Existing digital painting software enables to make some artworks including the scratch art. However, novice users are too hard to use the software. If users can color the scratch art easily, they may find their own artistic expressions and facilitate to participate in some art activities. This study presents a digital scratch art painting interface using a blob-based method proposed by Shugrina et al. The interface allows users to color by their sketches and then the gradient colors are generated automatically. Furthermore, users are possible to edit each color blob, in fact, every blob is movable and changeable its sizes. We also adopt a kubelka-munk model to mix colors based on real pigments. To verify the effectiveness of the proposed interface, we conducted the experimental evaluation. The experimental results confirmed that the usability of the proposed interface is superior to the existing painting software. This paper describes the details of the proposed interface and the experimental results.

In the H.265/HEVC video encoding standard, the reference software HEVC test model (HM) utilizes rate distortion (RD) optimization for intra prediction mode decision. Previously, there have been some trials to use neural networks to determine the prediction modes. However, networks with deep layers also pose a high calculation complexity. Therefore, a method has been proposed in this study to reduce the computational complexity by introducing two stage neural networks.

Many Super-Resolution algorithm using DNN have been proposed. They simply expand the image size. On the other hand, bit depth expansion has also been proposed independently. In this paper, we consider the case when the image enlargement process is combined with bit depth expansion by Super-Resolution processing. Experimental results show its validity in both SDR and HDR images.

We propose a speech motion feedback system for improving articulation by showing 3D-CG jaw and abstracted lip. The subject's motion of the jaw is captured by a 3D position with rotation sensor. The lip motion is measured by four 3D position sensors using infrared emission. Subjects observe his/her own face on LCD screen, and 3D-CG jaw and abstracted lip motion on semi-transparent screen. Subjects reported that they noticed the importance of the motion of the speech organs after the experiment.

Drosophila, as known as fruit fly, is a holometabolous insect typically showing four life-stages ― embryo, larva, pupa, and adult. A fruit fly is often used as a model animal in gene manipulation research since it is easily bred and manipulated with genetic operation. However, detection of the life-stages is manually performed in laboratories. In this paper, we propose the automatic life-stage detection system. We trained the neural network specializing in a semantic segmentation task. As a result, the network could detect pixel-wise an animal in a picture. Moreover, simple signal processing achieved precise detection of life-stage transition comparable with manual operation. It is expected that our system promotes high-throughput screening in genetic research manipulating drosophila.

Environmental education is educational activities using nature. Since the importance of environmental education is increasing, development of effective leaders training methods and learning programs is required. We consider that by using learner’s interests during educational activities for learning evaluation, we can improve the teaching methods of instructors and learning programs. In this research, we use gaze directions to measure interests. In outdoor such as environmental education, it is difficult to apply gaze estimation techniques using conventional cameras due to the effects of sunlight and fog. In addition, since the wearable device imposes a physical burden on the learner, it has a problem that natural state data cannot be measured. we attach a compact and lightweight device to a cap or helmet. In this research, we measure head direction and estimate gaze direction from the data. The head direction measurement system uses a transmitter with three ultrasonic speakers and receiver. We can measure horizontal and vertical head direction by received time difference of ultrasound transmitted from three speakers. By using the M-sequence for the transmission wave, the spectral efficiency increases, enabling simultaneous measurement of the head direction by many people. In the experiment, we measured horizontal head direction using two ultrasonic speakers. In the result, it was possible to identify the M-sequence and received time. The error in the head direction was 5 degrees.

It is difficult for the conventional Mask Regions with Convolutional Neural Network (Mask R-CNN)¹ to distinguish different objects with similar features of the shape. In this paper, we improve the object classification performance of Mask R-CNN by expanding the region proposal appropriately and using it for learning. The results of experimental evaluations using our modified 300-W dataset² show that the mAP of our proposed method is improved from 0.631 to 0.701, compared with the original Mask R-CNN.

Most artificial neural networks that originate from the perceptron use the inner product as the basic operation to calculate pattern similarities. Unlike them, we propose a novel hierarchical network model based on a pattern dissimilarity operation using a popular dissimilarity metric: sum of squared differences. Our model is differentiable and end-to-end trainable. We provide a description of the basic formulation and network architecture of the proposed method. Then we apply our method to image classification tasks using public datasets for performance comparison. Although our method does not outperform the same size of convolutional neural network in terms of classification accuracy, it demonstrates that comparable performance can be obtained.

The following three visual phenomena have been confirmed as a visual effect occurring at an edge having a density change: (a) Mach band, (b) Chevreul illusion, and (c) the Craik-O'Brien-Cornsweet effect. These are considered to be different phenomena. In our previous research, it was found that by varying the blurred edge width the density value of the image changed and the contrast decreased approximately by 20% at the maximum. From this phenomenon, it was possible to explain to some extent the relevance of the above three phenomena by the edge width. Furthermore, under the research of gazing time and edge blurring phenomenon, we confirmed that edges disappear with low contrast. In this paper, we focus attention on the gazing time and conduct an evaluation experiment on how much the Troxler effect occurs, using the density difference of the presented image as a parameter. We also investigate the evaluation of the Mach band at that time. Through these experiments, it is possible to show the experimental results that clarify the relationship between the four phenomena.

We described the evaluation method of stereoscopic images by using perspective images with continuity that is easy to feel a depth in 2D images. In this paper, we propose a new 3D evaluation method, using reversed images and an evaluation item of an incongruity. As a result, in terms of the perspective image, we got the effectiveness of the evaluation item “incongruity” in three evaluation items of depth feeling, stereoscopic feeling, incongruity.

Recently, as devices and communication technologies capable of acquiring digital images such as smart phones and digital cameras are developed, demand for digital images is increasing. However, the noise is generated in the process of acquiring, transmitting, and storing digital images. Therefore, in this paper, we propose an algorithm using an enhanced median filter and a Bitonic filter to reduce high-density impulse noise and preserve the edge region. Experimental results demonstrate that the proposed algorithm outperforms the conventional filtering techniques for high-density impulse noise.

In this study, we used a Kinect V2 RGB-Depth sensor to measure the three-dimensional shape of faces and calculated the information of the differences among several individuals. Based on Principle Component Analysis of the obtained information, we attempted to verify the possibility of generating various expressions from an inexpressive face of an arbitrary individual. Additionally, we attempted to improve the quality of generated utterance expressions.

Recently, there has been an increase in the use of video and statistical data in sports practice and coaching. A playertracking function plays an important role in order to obtain players’ trajectories which are essential for performing statistical analyses of players. Usually, in video-based tracking systems, sensor cameras are mounted to ceiling to prevent players from overlapping on the image, because the overlapping makes it difficult to detect overlapped further players, which may lead wrong trajectories. So, the ceiling cameras are useful for tracking but it is expensive to mount on ceiling, thus a system using wall-mounted cameras that can be installed at low cost is desired. However, wallmounted cameras cause a problem that players may overlap each other on the image. To overcome this problem which is inevitable for such wall-mounted camera systems, we propose a stable tracking method which determines the position of the player having less overlapping ratio earlier and determines ones having more overlapping ratio later with using previously determined players’ positions. The overlapping ratio is defined by using the positional relation between the target player and one’s surrounding players from all of those images where the target player exists. Experimental results with real basketball scenes show that the positional error is less than 15cm, and the processing time is less than 10ms/frame at 3.0 GHz CPU, which proved that our method has enough performance for using real-time player tracking system.

Unmanned aerial vehicles (UAVs) are applied to various applications due to its maneuverable flight. Formation flight composed of multiple UAVs has obvious advantages in accomplishing effectively and speedily a task. We propose a formation control of multiple UAVs for omnidirectional patrolling. The formation flight adopts a leader-follower structure. Assumed that motion capture system detects the 3D configuration of formation flight, our method stably and safely controls the formation based on geometric constraint in 3D space. When we fly the leader UAV by remote control according to given flight trajectory, the positions and orientations of follower UAVs are automatically adjusted by our control system so that the leader-follower structure retains the geometric formation. We demonstrate that our proposed method offers excellent formation flight to provide practical omnidirectional patrolling.

We aim to provide a large-scale point cloud-based 3D map that reflects the internal structure in a building for autonomous mobile robots. We propose a new method that iteratively updates the 3D map based on self-localization in dynamic environment. We assume that a patrol robot collects 3D points required for constructing a 3D map in database. We adopt multi-layer NDT (Normal Distributions Transform), which handles multiple horizontal and multiple vertical scan lines, to robustly estimate the robot’s 3D position in real environments. Our proposed method estimates a specified floor in the building and determines a 2D localization on the floor. Based on the self-localization, the method detects depth variations by taking advantage of 3D LiDAR. Once our method detects some dynamic changes on patrol, it replaces the previous 3D points corresponding to the space in 3D map with the latest 3D points. As our approach estimates an accurate self-localization in the 3D map, the 3D map updated by the method is seamless without giving uncomfortable feeling. We demonstrate that our iterative update method is an effective way of successively renewing the 3D map for inside a building.

We propose a new method for measuring three-dimensional (3D) estimation distribution in water around a heat source. By using near infrared (NIR) light, it is possible to measure the total amount of temperature on the optical path. The total amount of temperature obtained is converted into local temperature distribution based on certain assumption. The conventional method estimates temperature distribution using symmetric inverse Abel transformation based on the strong constraint that heat propagates axially symmetrically from the axis passing through the center of the heat source upward. There are two points to be improved in the conventional method. The first point to be improved is that the temperature is assumed to be distributed circularly symmetrically from the axis passing through the heat source center. In fact, in many cases, heat propagates asymmetrically. In this case, the conventional method cannot obtain accurate results. In this paper, we use asymmetric inverse Abel transformation instead symmetric inverse Abel transformation in order to perform asymmetric estimation. The second point to be improved is that the conventional method does not take into consideration that heat propagates smoothly. In this paper, we use an optimization algorithm to smooth adjacent surfaces of the temperature distribution. Experiment showed that the proposed method estimated temperature distribution more accurately than the conventional method.

In this paper, we propose a weight calculation method to solve a problem of Multiscale Normalized Cut (MNCut) image segmentation. Since Multiscale is incorporated with various information, robust segmentation results can be expected for various RGB images. However, MNCut has a problem that if the foreground or background is separated into two or more regions, they may be classified into different labels. To solve this problem, we use Matting Laplacian2 for edge weight calculation. In the experimental results, the proposed method obtained the results closer to the ground truth.

This paper proposes a novel image contrast enhancement method based on both a noise aware shadow-up function and Retinex (retina and cortex) decomposition. Under low light conditions, images taken by digital cameras have low contrast in dark or bright regions. This is due to a limited dynamic range that imaging sensors have. For this reason, various contrast enhancement methods have been proposed. Our proposed method can enhance the contrast of images without not only over-enhancement but also noise amplification. In the proposed method, an image is decomposed into illumination layer and reflectance layer based on the retinex theory, and lightness information of the illumination layer is adjusted. A shadow-up function is used for preventing over-enhancement. The proposed mapping function, designed by using a noise aware histogram, allows not only to enhance contrast of dark region, but also to avoid amplifying noise, even under strong noise environments.

This study proposes an interactive image segmentation method based on high dimensional self organizing maps (SOMs). The proposed method was applied to gray-scale and color images. The experimental results demonstrated that higher dimensional SOMs were able to achieve more accurate segmentation.

Stereo matching methods estimate the depth information from stereo images using the characteristics of binocular disparity. After they find corresponding points between the left and right viewpoint images, the disparity value of two corresponding points is calculated. Since stereo matching methods search for corresponding points based on features in the stereo images, it is possible to estimate the accurate disparity value in the textured region. On the other hand, the inaccurate disparity value is estimated in the textureless and occluded regions and it causes disparity errors. The guided image filter is used for the disparity error refinement. However, the disparity refinement using this filter causes blurred pixels near the edge region in the refined disparity map. Therefore, we propose a post-processing method to alleviate this problem. The proposed method detects disparity errors in the initial disparity. After they are detected, we refine disparity errors using the modified guided image filter that applies the joint bilateral filter to the coefficient smoothing step in the guided image filter. As a result, the proposed method enhances the initial disparity map by alleviating the blurring artifact near the edge region in the disparity map.

The h-Index is frequently used to evaluate researchers. However, the index is not suitable for young researchers because they typically have only a few prior achievements. In this paper, we evaluate the potential of JSPS research fellows using the h-index and the betweenness centrality of co-author networks. The results show that the new measure appropriately evaluates the potential of young researchers at their early stage, as compared with the h-index.

In stomach lesion screening, endoscopic images provide the most effective diagnostic information. However, in the most of lesions at the initial stage, the sign of existence is hard to appear on endoscopic images, and also there is the difference in operations of endoscopes and observation of images in real time among individual medical doctors. Therefore, development of a computer aided diagnostic system (CAD system) for endoscopic images is required. In this study, we propose a method for automated detection of fundic gland polyps from endoscopic images using an object detection algorithm named SSD (Single Shot MultiBox Detector) which is one of CNN (Convolutional Neural Network). SSD used here has 20 of convolution layers and 6 of pooling layers, and the input image size is 300x300. In the experiment, 73 practical fundic gland polyp images were used. To compensate for lack of training images, augmentation was performed using image rotation and edge enhancement. We trained 8751 training images and 2188 verification images. Also, as a preprocessing, highlight areas were removed automatically from all images including both training and test samples. As a result, 94.7% of TP (true positive) rate for 73 fundic gland polyp images was obtained by using our learned SSD.

In general, there is a service that needs to correspond "anytime" "anywhere". However, fault-tolerant systems with fully redundant hardware used for business are very expensive and cost effective. In addition, problems such as occurrence of down time of the entire system due to maintenance and switching to a backup circuit occur, and it is a problem that it is difficult to secure a bypass route. In this paper, the method to construct a fault tolerant system at low cost only by software processing is proposed. Especially focused on video processing and transmission. Even if arbitrary images are simultaneously displayed by a plurality of processes as the characteristics of the image, the influence on the entire display is not very large. Therefore, images simultaneously processed by a plurality of pipelines are collectively displayed. As a result, the pipeline in which a series of moving image processing is processed without stopping, is constructed. As a basic system, the system that duplicated simple image processing, and confirmed that there is no problem even if one side stops, is constructed.

Active stereo is one of the means for estimating depth, which uses a camera and a projector. The projected patterns are classified into two groups: a time code or a space code. In recent years, a space-time hybrid code was investigated to fully utilize the advantages of both codes. With this method, one needs to separate the information from captured images, on which the information for both codes are coexisting and mixing. In this paper, we propose to use the color space to successfully construct and extract the information for the time/space codes.

To prevent subarachnoid hemorrhage that causes serious damage to the brain, clipping and coil embolization are used in Japan, while liquid embolization is also used in overseas. Liquid embolization is an effective technique that can treat with a distorted and enormous cerebral aneurysm. However, it is not authorized yet in Japan because the embolic material flows out of the aneurysm, and embolizes normal blood vessels too. Then, as a preliminary simulation to verify the safety of liquid embolization surgery, we have been developing a particle-based high precision liquid injection simulation with CFD (Computational Fluid Dynamics), and have verified the accuracy by comparison between the simulation results with the corresponding physical experiments. The simulation size, however, was smaller than the physical experiment, since larger size simulation needs huge amount of memory, which cannot be treated with a normal PC. As the result of the simulation, a droplet was formed, but contacted the side surface of the water tank that imitated an aneurysm, and the quantitative assessment could not be performed. Therefore, this paper describes the simulation result with a larger size of environment, and the quantitative assessment by comparison of the aspect ratio of the droplet between the simulation and the physical experiment. In the assessment, we have confirmed that the shape of the droplet formed in the simulation is similar to the one of the physical experiment.

In this paper, we propose a welding skill training system for beginners using VR technology. In this system, a user can do welding in a virtual space in some situations, and a user can also observe welding operation by an expert from any point in a virtual space. Experimental result suggested that this system would be effective for welding skill training for beginners.

In recent years in Japan, the labor shortage has become serious problem in various fields of industry because of declining birthrate and aging. Among them, welding work requires training and qualifications to master the welding work, and it is one of the areas where personnel shortage is prominent. Therefore, utilization of information technology for welding skill training is expected. In this paper, we propose to utilize AR technology for welding skill training. We developed a system which can visualize the posture during welding and show a guide for welding in real space by using AR technology for training and supporting welding works.

We discuss material properties of different garment wrinkles. Subjects wear three kinds of different fabric garment: polyester, cotton, and linen. Scanning the subject's 3D body shape using a RGB-D camera and getting a 3DCG model of them. Generating a normal polygonal image applying a baking process. High-poly mesh is generated from low-poly mesh of the 3DCG model, and multiple high polygons are mapped into a low polygon. Computing the normal vectors of the high polygons on the low-poly mesh, generators our normal map image through this process. Garment wrinkles are detected by straight-line recognition on the normal map using Hough transform. Measuring the number of the garment wrinkles, the variation in the direction and position of garment wrinkles, and the length distribution . All properties of the garment wrinkle is necessary for an analysis in our experiment. But it is revealed that the dispersion in the wrinkle direction is important for distinguishing the fabrics.

Optical flow estimation in onboard cameras is an important task in automatic driving and advanced driver- assistance systems. However, there is a problem that calculation is mistakable with high contrast and high speed. Event cameras have great features such as high dynamic range and low latency that can overcome these problems. Event cameras report only the change in the logarithmic intensity per pixel rather than the absolute brightness. There is a method of estimating the optical ow simultaneously with the luminance restoration from the event data. The regularization using the L1 norm of differentiation is insufficient for spatially sparse event data. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical ow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical ow becomes radial from the FOE excluding the rotational component. Using the property, the optical ow can be regularized in the correct direction in the optimization process. We demonstrated that the optical ow was improved by introducing our regularization using the public dataset.

In this paper, we propose an interactive projection mapping using multiple chairs. In this content, images are projected from the celling to the seat of the chairs. The projected image would change depending on the arrangement of the chairs interactively. When the chairs are placed apart, a small image is projected on the seat of each chair. When the chairs are gathered, a large image is projected on the entire set of the chair seat. We implemented a miniature prototype system and experimented.

Several studies for layout design optimization depend on evaluation indices with necessary passage, spaciousness, etc. It is difficult to obtain a friendly layout by using conventional methods. The layout design-aided model in this study is a residence space, and there are eight types of furniture. All furniture is first allocated to each room by using a genetic algorithm. All allocated furniture’s initial arrangements in each room are then determined by using Q-learning. A user checks the initial layout through virtual reality and evaluates it subjectively. The layout for a specific user is flexibly fixed by applying Q-learning, and a user subjective reward is added. As a result of observer experiments, more than half of the furniture can be arranged in an ideal position for a user, and a satisfactory layout is successfully generated.

In order to organize the event such as ceremonies, lectures and academic societies, we have to make plans such as layout of the venue, placement the staff, route of entry and so on. In addition, it is also important to make sure of the lighting and sound effects of the venue, and to carry out simulations assuming troubles. On the other hand, a presenter is likely to be in trouble caused by being unfamiliar with the venue. It is also important to confirm the situation of the venue and the position of the seat as participants. However, from the point of the venue reservation and movement cost, it may be difficult to preview and rehearse at the venue several times. Therefore, the purpose of this research is to develop a stage event support system using virtual reality techniques. By using this system, organizer can simulate placement of the venue and simulate the schedule, presenters can confirm the environment of the venue and practice, and participants can check the situation of the venue. When a system user wears a head mounted display (HMD), she/he can perform rehearsals and simulations before the event such as confirming the environment and practicing events as if she / he is in the real space on.

As an effective reconstructing technique to observe full-parallax holographic images in the wide viewing area, we have studied a projecting process of 3-D holographic images with the use of the spatial screen. Last year, as one technique to perform a suitable projection of the images, we have reported a projecting process of holographic 3-D images onto the underwater micro-bubble screen. Applying this technique, the stability of the images displayed in the screen seems to have been highly improved, however, as for the flickering of the images, much more improvement seems to be not so easy. Moreover, it requires a lot of water in the water tank, so that a complicated and a more weighty system has been needed. For this reason, we constructed a new spatial screen system employed the fog, in which a complicated structure and a lot of water was not required. In this report, we have discussed its effects to the reconstructed images through the holograms. In addition, in this study, we have seen that the system construction seems to be simple and not so weighty, and effective for suppressing the flickering of the images with higher resolution.

We proposed an efficient coding scheme for a dense light field, i.e., a set of multi-view images taken with very small viewpoint intervals. The key idea behind our proposal is that a light field is represented only using some weighted binary images. This coding scheme is completely different from modern video codecs, but it has some advantages. For example, the decoding process is extremely simple, which leads to a faster and less power-hungry decoder. Moreover, we found that our scheme can achieve comparable rate-distortion performances to that of modern video codecs for datasets with small disparities. However, it is difficult to express regions with larger disparities using only a limited number of common binary images. Therefore, in this paper, we extend our light field coding scheme to the one with disparity-compensation and show its effectiveness.

In general, "drawing collapse" is a word used when very low quality animated contents are broadcast. For example, perspective of the scene is unnaturally distorted and/or sizes of people and buildings are abnormally unbalanced. In our research, possibility of automatic discrimination of drawing collapse is explored for the purpose of reducing a workload for content check typically done by the animation director. In this paper, we focus only on faces of animated characters as a preliminary task, and distances as well as angles between several feature points on facial parts are used as input data. By training a support vector machine (SVM) using the input data extracted from both positive and negative example images, about 90% of discrimination accuracy is obtained when the same character is tested.

We previously proposed a machine learning based post filtering method for reducing image artifacts caused by lossy compression. The method classifies reconstructed image samples into three categories using a support vector machine (SVM) to roughly discriminate magnitude of the reconstruction errors. Then, an optimum offset value is added to the samples belonging to each category in a similar way to the post filtering technique called sample adaptive offset (SAO) used in the H.265/HEVC standard. In this paper, two kinds of SVM classifiers are adaptively switched according to information on block boundaries of transform units (TUs) in H.265/HEVC intra-frame coding. Furthermore, samples used for a feature vector, which will be fed to the SVM classifier, are rotated at the block boundary to properly capture local characteristics of the reconstruction errors.

Elucidation of information processing in our brains is progressing given the highly informationoriented world we live in. Noise is inevitably present in both man-made and natural systems. Previously, these elements were removed for signal detection and information processing. However, recent studies have reported that noise plays a major role in brain information processing. One of the salient features of the relationship between noise and the vision system is the stochastic resonance phenomenon, wherein the detection rate of a weak signal is improved by the visual addition of a blinking noise of appropriate intensity. Improved understanding of the vision system is very useful for the development of imaging technology. This strategy of improving weak signal detection can be applied to digital image processing. In this study, we propose a vision model based on the FitzHugh–Nagumo equation and confirm that the stochastic resonance in brightness perception can be described by the model.

We have been investigating a novel depth estimation system that adopts tilted-lens optics for real-time usage, e.g., automotive tasks. Herein, we obtained depth values for each pixel from the sharpness ratio of only two tilted optics images; we used a monocular camera system with a spectroscopic mirror. However, the method causes some estimation errors because of the difference between the optical theory and the actual camera system. Therefore, to reduce the error, we adopted a neural network to obtain the depth map. In this paper, we report our improvement by optimizing the neural network construction which calculates the depth value for each pixel from 3 × 3 pixel values at each image and y-coordinate.

Total-variation-minimization-based JPEG decompression is recognized as a promising technique for providing artifact-free images. Such a technique is typically accomplished in solving an optimization problem based on iterative methods. However, conventional decompression techniques sometimes exhibit over-smoothed regions in decompressed images by increasing iteration count. As a result, users are confused since an optimal iteration number is unknown for an arbitrarily given JPEG bitstream. To overcome the difficulty, we propose a new iterative JPEG decompression technique that combines a total variation minimization strategy with the so-called compressed sensing. In contrast to existing approaches, our technique always provides optimal decompression results, despite increasing iteration count. The experimental results indicate that the proposed technique is advantageous over conventional ones.

This paper proposes a block adaptive CNN/HEVC prediction method for bidirectional motion compensated prediction which is one of the key technologies for video coding. We have already proposed a deep convolutional neural network (CNN) model which can predict the target block to be predicted from its spatially co-located blocks existed in the temporally previous and future frames. The CNN model estimates four geometric transformation matrices, and the predicted values are produced by transforming both previous block and future block using them. We have demonstrated that the method is greatly effective for frames with complicated motion. However, the conventional CNN model often did not work well for the frames with large motion. In this paper, we introduce an adaptation method which selects the one of CNN based prediction and HEVC based prediction. Experimental results show that the prediction error using the proposed method can be reduced to 60% to 90% for many kinds of videos with complex motions compared to only HEVC prediction.

This paper presents a physics-based method for simulating the phenomenon of ink diffusion in fluid. Here are two contributions in our method: 1) Our method is physics-based method, which can simulate ink diffusing phenomenon realistically. 2) CUDA acceleration and spatial subdivision optimization is applied to our method, thus our method can obtain a realistic simulation result in real time. This method is a grid-particle combined method: We use two different particle systems to represent fluid and ink, and use a improved smoothed particle hydrodynamics(SPH) method to simulate the behavior of fluid, velocities of ink particles are calculated by neighbor fluid particles. In our method, Compute Unified Device Architecture(CUDA) acceleration technique and the space subdivision optimization are applied to achieve a real-time simulation. The rendering of ink particles uses point sprites and blurring techniques. Compared to other ink simulation methods, our method is more realistic and efficient.

Motion estimation (ME) has the highest computational complexity in motion-compensated frame rate up-conversion (MC-FRUC). For the real-time implementation of FRUC, a fast ME algorithm is required. In this paper, a new hierarchical ME algorithm for MC-FRUC is proposed. It constructs an image pyramid by dividing the frame into several sub-images according to resolution, and performs ME at the top level to reduce complexity while improving accuracy by selecting multiple motion vector candidates. These candidates are refined at the lower levels, and the final motion vector is selected at the bottom level. Thus, the proposed algorithm obtains an average peak signal-to-noise ratio gain of upto 0.85 dB compared to conventional algorithms with lower computational complexity and yields interpolated images with better visual quality than other methods.

Ultrasound imaging is applied to various fields because it is noninvasive and real-time imaging is possible. However, in the field of diagnosis imaging, ultrasound imaging is inferior in resolution to other modalities, so researches to improve resolution are actively conducted. We proposed a method called the SCM (Super resolution FM-Chirp correlation Method) that improves the range resolution based on frequency sweep. In order to improve the frame rate, the SCM was extended to the SA-SCM (Synthetic Aperture-SCM) by incorporating synthetic aperture beamforming using divergent waves in the SCM. However, these methods improve only the range resolution. Therefore, in this study, we aim at high lateral resolution by using F-DMAS (Filtered-Delay Multiply and Sum) of analytic signal representation as the beamforming method.

In this paper, we propose using image recognition techniques to estimate the “understanding measure” in person-toperson teaching situations. The phrase “understanding measure” refers to how strongly a teacher feels a student understands a topic. First, we extract a student’s nonverbal behavior (head movement, gazes, and blinking) as the features for the estimation process. Next, we calculate the subspace from the aforementioned feature by using principal component analysis (PCA) and linear discriminant analysis (LDA). Finally, we classify unknown data as either “understood” or “did not understand” by using a kNN classifier in subspace. Our experiments confirmed that the Fmeasure of the classification “understood” by our method was 0.75 and “did not understand” was 0.60, indicating that our method improved F-measures 0.38 and 0.11, respectively, compared with previous methods.

In this paper, we propose a method to analyze the differences in the motions of gaze and hands between skill levels in assembly tasks. The method quantizes the positional information of the gaze and hands into eighteen areas and converts the positional information of into a code. Next, it calculates the occurrence frequency of pairs of codes. Then, it generates co-occurrence histograms, called “Gaze/motion integration features,” using the occurrence frequency. We can analyze the relationship of motions between the gaze and hands using these features. The results of an analysis of a skill improvement process are that the non-dominant hand of an “elementary level” stays in two areas, and the non-dominant hand of an “intermediate level” moves to five areas. Therefore, humans can move their non-dominant hand more efficiently at the “intermediate level” than at the “elementary level.” In addition, we found that the gaze of the “intermediate level” moves to eight areas, and the gaze of the “expert level” moves to three areas. Therefore, we found that the gaze of the “expert level” remained at the center of the workbench.

The aim of style transfer is giving the style from one picture to another. The application of neural network in image processing separates the high level features and low level features of the image in the process of style transfer, and derives a variety of methods and optimization for style processing. The style transfer generates new images by separating and recombining the content and style of original images. In this process, various factors such as color and illumination will affect the result. The traditional algorithm only focuses on continuous pixels and the whole image, this paper will extend the process object to the contour of the image, and improves the detail processing from the existing style transfer examples. From the contour of images, the target image retains the contour feature of style image and the content of original image, in other word, gives the contour style of style image to original image. Finally, the style transfer effect based on the original image contour is obtained with some defects. The work can be easily extended to the aspects of video and 3D images.

In the color imaging, single sensor system with color filter array (CFA) is popular, especially with Bayer CFA. Performances of de-mosaicking for Bayer CFA are improved by using correlation of color planes, and, evaluated by reconstruction of color plane on still images. However, captured images is used for video coding recently. Therefore, it should be tested including video coding. We have proposed an imaging suitable for video coding. In this method, colors of CFA are changed to pastel color, also, the color plane reconstruction method that abstract the average component of RAW image are proposed. In this, the color of CFA is simulated by computer calculation. In this paper, we propose color plane reconstruction method using the average component and the color difference. In the experiments, we evaluate the reconstruction performance of RGB images and the overall performance include video coding. To evaluate coding performance, Y-PSNRs and coding bitrate between the original images and the decoded images are used. From the experiment, Y-PSNRs of proposal have the advantage. With the coding, the bitrate reduction is up to 30% at the same PSNR, because of less alias component.

We can generate arbitrarily focused images with arbitrarily aperture shapes from a light field, which is usually captured with a specific device such as a light field camera. In contrast, we propose a method that only needs a sparse focal stack, i.e., a few differently focused images, which can be easily captured with a conventional camera, for the same purpose. Specifically, our method first reconstructs the underlying dense light field from the sparse focal stack, and then, generates arbitrarily focused images.

In this paper, we propose a method of object tracking using viewing directions of multiple spectators' cameras. Attitude sensors in a smartphone can measure its shooting direction on time. By collecting the directions and analyzing the overlapping situation of multiple view fields, a remarkable part of the scene situation can be extracted as the most overlapped area of them. We implemented the analysis by using the blending function with a configuration of interested degree map in the view fields. We adopted it to actual sport scene situations, also, we got good estimation results close to the ground truth.

Three-dimensional computer graphics (3DCGs) of disaster scenes can be created to visualize the situation of the disaster areas. Our system is based on the Structure from Motion (SfM) technology with collected photographs or videos taken with digital camera or video camera. Moreover, it is possible to write information onto the surface of these 3DCGs through the handwriting with a digital pen and the dot screen; the 3DCGs are projected with a projector on the dot screen. The 3DCG created from images taken by victims or rescue teams is a useful tool for disaster relief. This paper presents the simulations that took place in the disaster experience facility “Sona Area Tokyo of The Tokyo Rinkai Disaster Prevention Park” and “921 Earthquake Museum of Taiwan in National Museum of Natural Science.”

Bare-handed interaction in augmented reality (AR) experience contributes to realizing a natural interaction with a virtual object. The difficulty of the bare-handed interaction is that there is no physical feedback. Our AR system presents auditory stimuli when a user grasps and releases a virtual soft object with a bare hand. We examine effects of auditory stimuli on the perception of the pseudo-softness of a virtual soft object and the ease of handling it.

This paper proposes a novel approach for smoke matting and compositing for video sequences. The proposed method is based on the assumption that nearby pixels have similar foreground colors and alpha values. The matting does not require user interaction, nor a physics-based model for smoke. Moreover, even multicolored smoke can be matted, since no constraints are placed on foreground color. Meanwhile, the compositing can modify smoke colors by linear color transformation. Experimental results show that the proposed method can matte the smoke consisting of multiple colors and alter the colors without impairing natural feeling in the compositing step.

In this paper, we propose a new color quantization method that can preserve infrequent salient colors of an original image. The infrequent salient colors mean that they are not dominant globally, but are dominant locally and are important to keep the impression of the original image. In the proposed method, color quantization is realized by k-means clustering and an input dataset for the clustering are adaptively and repeatedly modified based on local quantization errors to preserve the infrequent salient colors. The proposed method is implemented as an Android application to verify the feasibility of the use of the proposed method on mobile devices.

This paper describes a new method of image registration using distortion-tolerant template matching via multi-scale subwindow search. Here, we make full use of the GPT (Global Projection Transformation) correlation technique that maximizes a normalized cross-correlation value between an optimally 2D projection transformed template and a subwindow area of an input image. In particular, we propose to adaptively change the shape of the subwindow area from an original rectangle to its 2D projection transformed one through iterative matching process via the GPT correlation. We name this algorithm: adaptive subwindow control. Experiments made on the well-known datasets, Graffiti and Boat, show that the proposed method achieves a far superior ability of image registration under varying zoom, rotation, and viewpoints to the well-known feature-point based technique: a combination of ASIFT (Affine Scale-Invariant Feature Transform) and RANSAC (Random Sample Consensus).

A computer vision computation requires high number of multiplications causing a bottleneck. Based on the work of Zhenhong Liu, the multiplications in these algorithms do not always require high precision provided by the processors. As a result, we can reduce computation redundancy by means of multiplication approximation. Following this approach, in this paper, we investigate two major algorithms namely convolutional neural network (CNN) and scale-invariant features transform (SIFT) to find their error tolerances due to multiplication approximation. A multiplication approximation is done by injecting a random value to each of precise multiplication value. The INRIA and OXFORD datasets were used in the SIFT algorithm analysis while the CIFAR-10 and MNIST datasets were applied for the CNN experiments. The results showed that SIFT can withstand only small percents of multiplication approximation while CNN can tolerate over 30% of multiplication approximation.

The authors have proposed a novel depth from defocus (DFD) technique based on a cross reblurring approach. While the technique gives a precise depth estimation, it is unfortunately affected by a disalignment of pixels in a set of multi-focus images. This paper thus proposes a compensation technique for the disalignment to reduce the estimation error. The experimental results given in this paper provide the validity of our approach.

In this paper, we propose codec classification algorithm based on recurrent neural network (RNN) model. In video compression, codecs, such as MPEG2 and H.264/AVC, have their own distinctive data structure. These unique structures which are almost shown in header can be considered their feature. The proposed algorithm exploits that characteristics for classifying unknown bitstreams into specific codec. According to the fact that RNN is appropriate to time series data for learning to classification/recognition, the feature of an encoded bitstream can be extracted. We constitute the encoded bitstream as an input and give the bitstream its label indicating codec index. Two standard codecs, MPEG2 and H.264/AVC, are used in experiment. Experimental results show that the proposed RNN model classified bitstreams into corresponding codecs to some extent.

In recent years, 360 videos have been attracting increasing attention as a new type of video that provide immersive experience. Joint Video Experts Team (JVET), which is developing next generation of video coding standard called Versatile Video Coding (VVC), also deal with the compression of 360 videos in the scope of the standard. In the workflow of 360 video coding of JVET, firstly the 360 video is projected onto the 2D plane with a projection format. Some projection formats have inactive region in the converted 2D plane such as rotated sphere projection (RSP). This paper proposes methods of inactive regions padding to improve coding efficiency and reduce visual artifact. The proposed method of inactive region padding gives -0.07% and -0.07% average coding gain of BD-rate in terms of endto-end S-PSNR and WS-PSNR, respectively. In addition, the visual artifacts along the edges of discontinuous faces could be reduced.

Optical see-through (OST) head-mounted display (HMD) enables the user to see the real-world and virtual objects together. Previous OST HMD-based augmented reality applications mainly attempt to render the virtual objects that are geometrically and photometrically well-aligned with the real-world. In this paper, we show that the user's perception of the real-world can be changed by overlapping the image of the real-world with the real- world. As a case study, we apply image refocusing to the captured real-world image and display the refocused image via the OST HMD such that the user sees the real-world and refocused image together. To this end, we first perform an experiment to find the relationship between the synthetic blur of the augmented object and the perceptual blur of the real-world object. We then apply the depth-adaptive image defocus blur to the image of the real-world to create the perceptual blur of the real-world. Experimental results show that the proposed method can make special visual effects to the real-world that the user sees through the OST HMD.

In video-based light field coding, sub-aperture images (SAIs) are ordered to form a pseudo video sequence, and the sequence is encoded by a video compression algorithm, for example, by HEVC. When the size of SAI is not divisible by the minimum size of coding tree unit, proper boundary handling method is required. This paper investigates several boundary handling methods. To maintain high quality of the central SAIs, we combine rotation and u scans to have a new hybrid scan order. Random access configuration is used instead of low-delay one for better coding efficiency. The proposed methods are evaluated with the latest coding tool.

In this paper, we propose a land-cover classification method based on a modified hierarchical k-nearest neighbor (MHkNN) algorithm to achieve a high classification accuracy. The proposed method introduces a reliability measure for each training sample, which is defined as confidence in the sample belonging to each of the considered classes. The method performs the majority voting considering not only the number of the training samples, but also their reliabilities. The classification performance of the proposed method is compared to that of the conventional land-cover classification methods. The effectiveness of the proposed method is verified by applying it to real remote sensing images.

This paper proposes a blind detection of fabric defects using multiple image features. The aim of proposed method is to detect the new types of fabric defects that have not been learned. For general learning of image features, this paper first learns the characteristics of image features between normal and defective image patches for various types of fabric structures and defects. The image features are frequency coefficients, color histogram, and edge orientation histogram. The mean vectors of 3 image features are calculated, and the vector distance distributions of normal and defective patches are learned using support vector machine. Since the decision boundary is determined from general distributions of image features, the proposed method is not restricted to the detect types and fabric structures which have not used in the learning phase. According to experiments with the real fabric images and defects types that have not been learned, the proposed method detects fabric defects with 96.4% accuracy at 0.4% failure including false positive and false negative errors. This result outperforms the usual CNN (Convolutional Neural Network) approaches.

In this paper, we propose a method of generating a panoramic image sequence from a video sequence taken by a handheld camera. The panoramic image can be made by projective transformation with a homography matrix estimation for each frame using corresponding of image features. However, it is difficult to get a precise result because of blurring on the video by handheld operations, and movements of shooting objects in the scene. And the sensor information has several errors. Therefore, we consider using attitude sensor information together to compensate for the failure of embedding a frame onto the panoramic image. We explain a method for it and show several experimental results with considerations.

Video stitching technology is widely used to generate Ultra Wide Vision (UWV) and 360VR. However, preserving spatial-temporal coherence and reducing parallax are still challenging problem. In order to increase stitching quality, we propose a parallax-tolerant real-time video stitching method for 12Kⅹ2K UWV. Our experimental results show that the proposed method reduces the parallax efficiently.

Image segmentation is one of the most important techniques in computer vision and image processing. Many image segmentation methods have been proposed for these few decades. Hierarchical Feature Selection (HFS)1 is a graph-based approach for the image segmentation. It is known as a fast segmentation method that merges over segmented regions hierarchically. At the first level of the merge, the superpixels are utilized to obtain the over segmented regions. However, HFS sometimes fails when it is applied for the textured regions. In this paper, we propose a new approach for image segmentation, Searching Tree Segmentation from Superpixel (STSS), by formulating the merge of superpixels as a path searching problem. We construct trees and search the trees whose nodes correspond to the boundary of the superpixels and values of the nodes correspond to the distance between superpixels. Our algorithm does not check the boundaries of similar superpixels if these are no neighboring boundaries of the distinctively different superpixels to prevent the over segmentation of the textured regions, while HFS checks all boundaries including quite similar superpixels.

Being motivated by the Saak (Subspace approximation with augmented kernels) transform, we propose an image recognition scheme using multi-layer sparse feature extraction with a convex solver ADMM (Alternating Direction Method of Multipliers). The Saak transform consists of a multi-layer PCA (Principal Component Analysis) and S/P (Sign-to-Position) conversion to avoid sign confusion. This paper adopts sparse representation instead of PCA and also compares the S/P conversion with the activation function ReLU (Rectified Linear Unit), which is realized by involving the projection mapping onto the non-negative set in convex formulas. The Saak transform uses PCA not only for feature extraction but also for dimension compression of feature vectors. We expect that our method does not need the dimension compression since sparse representation compresses features more than PCA. Experimental results on the MNIST and Fashion-MNIST dataset show that the proposed method is equivalent to the Saak transform in recognition accuracy, and that our method can make features more sparse and extract features that have high discriminant power locally.

Multi-exposure image fusion is a method for producing an image with a wide dynamic range by fusing multiple images taken under various exposure values. In this paper, we point out two issues regarding color distortion included in fused images, that conventional fusion methods have not considered, and a novel multi-exposure fusion method is proposed to improve the issues. The first issue is that multiple images taken in the same scene have different colors. The second is that the color of fused images is not the same as those of the input images due to the influence of fusion functions. The proposed method enables us to preserve the hue of input images, called pure-color, while maintaining the wide dynamic ranges produced by conventional fusion methods. In addition, the proposed method can be applied to any existing fusion methods to improve the quality of images produced by the fusion methods.

This paper tries improving image recognition accuracy with Convolutional Neural Networks (CNNs). CNNs are one of state-of-the-art image recognition frameworks, and have used the Rectified Linear Unit (ReLU) as the activation function. However, the ReLU rectifies negative values to zero. This paper applies the Sign-to-Position (S/P) format conversion after convolutional procedures to eliminate the rectification loss. Experimental results show that the proposed method improves the recognition accuracy of the MNIST and Fashion-MNIST data set by 0.50% and 1.30% compared with a conventional CNN respectively. The S/P format conversion also contributes to negative image recognition, and results in 12.58% and 3.66% higher accuracy.

The goal of this research is to apply the state-of-the-art deep learning approach to human fall-down event detection based on Motion History Images (MHI) from multiple color video sequences captured at different viewing angles. MHI is derived by detecting and combining temporal 2D human contours from surveillance cameras. A human action can then be represented by several continuous MHI images. We then use deep learning approach (CNN + LSTM architectures) to recognize the fall-down behavior from MHI sequences. Our method is capable of not only recognizing the actions of walking, standing, falling down, but also rising after falling down to avoid excessive false alarms. The accuracy of classification into the above 4 short-term actions is capable of achieving 97.66%. We also compare the performances of deep learning architectures that use simple CNN or CNN+LSTM, one or two-stage training, and single or two cameras. Our contributions lie on two aspects: (1) improving the performance on human action recognition based on MHIs and a combination of CNN+LSTM architecture, (2) preventing the false alarm of falling-down events that actually need no help.

In recent years, along with the aging of society worldwide, decrease of working population has become a serious problem. For this reason, robots are expected to substitute human work, automate distribution, and support human daily life, especially, elderly care assistance. In this research, we focus on the support of those who need care in everyday life, and, in this paper, we propose a human-robot cooperative system that supports acquisition of objects in cooperation with a human. The outline of the action of this robot is as follows: (i) It moves to the location designated by a user autonomously (not by remote control): (ii) On arrival, the robot exchanges information by video with the user who is at a remote place, and acquires the objects designated by the user among those placed there. (iii) After the acquisition, the robot moves again autonomously to the user and hands over the objects to the user. In this paper, we focus on step (ii) and show methods and some experimental results.

This paper describes a method for creating cel-style CG animations of waving hair. In this method, gatherings of air are considered as virtual circles moving at a constant velocity, and hair bundles are modeled as elastic bodies. Deformation of the hair bundles is then calculated by simulating collision events between the virtual circles and the hair bundles. Since the method is based on the animator's technique used in creation of the traditional cel animations, it is expected to suppress a feeling of strangeness that is often introduced by the conventional procedural animation techniques.