Redirected Walking (RDW) technique allows users to walk indefinitely in a limited physical space while keeping the feeling of real walking. Developers use RDW controllers to manage RDW techniques based on physical and virtual environment information. However, traditional RDW controllers still suffer from many problems. For example, generalized controllers are less optimized, and scripted controllers are difficult to handle unexpected movements. Based on reinforcement learning, we present a novel RDW controller that allows the user to explore complex and large virtual environments while minimizing the number of collisions with obstacles in the physical environments. Our RDW controller directly prescribes the translation, rotation, and curvature gains by analyzing real-time information of the physical environment. The simulation-based experiments show that our controller significantly reduces the number of resets caused by collisions between user and obstacles of physical spaces compared to steer-to-center (S2C) and current state-of-the-art controllers using reinforcement learning.
With the rapid development of 3D capture technologies, point cloud has been widely used in many emerging applications such as augmented reality, autonomous driving, and 3D printing. However, point cloud, used to represent real world objects in these applications, may contain millions of points, which results in huge data volume. Therefore, efficient compression algorithms are essential for point cloud when it comes to storage and real-time transmission issues. Specially, the attribute compression of point cloud is still challenging owing to the sparsity and irregular distribution of corresponding points in 3D space. In this paper, we present a novel point cloud attribute compression scheme based on inter-prediction of blocks and graph Laplacian transforms for attributes residual. Firstly, we divide the entire point cloud into adaptive sub-clouds via K-means based on the geometry to acquire sub-clouds, which enables efficient representation with less cost. Secondly, the sub-clouds are divided into two parts, one is the attribute means of the sub clouds, another is the attribute residual by removing the means. For the attribute means, we use inter-prediction between sub-clouds to remove the attribute redundancy, and the attribute residual is encoded after graph Fourier transforming. Experimental results demonstrate that the proposed scheme is much more efficient than traditional attribute compression schemes.
High-quality depth information is urgently required with their increasingly wide application in many real-world multimedia fields. However, due to the limitation of depth sensing technology, the captured depth map in practice usually owns low resolution and poor quality, which limits its practical application. As we all know, consistency between high-quality color images and low-quality depth maps achieves good effects in depth super-resolution. But the edge inconsistency also limits the recovery of depth map. Inspired by the geometric relationship between surface normal of a 3D scene and their distance from camera, we discover that there are more consistency between surface normal map and depth map in the edge areas. Meanwhile, surface normal map can provide more spatial geometric constraints for depth map reconstruction, for both of them are special images with spatial information, which we called 2.5D images. In this paper, we propose a unified framework, Normal Data Guided Depth Map Restoration with Edge-Preserving Smoothing Regularization (NDEPS) method, via joint spatial domain and gradient domain regularization, one characterizing the relationship between surface normal data and depth in the spatial domain and another edge-aware constraint in the gradient domain. The proposed NDEPS method formulates a constrained optimization problem that can be solved by an iterative conjugate gradient(CG) algorithm. Extensive quantitative and qualitative evaluations compared with state-of-the-art depth recovery methods show the effectiveness and superiority of our method.
With the fast development and application of virtual reality (VR), panoramic images have been a hot research topic and been widely applied in many areas. Usually, the panoramic images are generated by image stitching with images captured by cameras set in 360 degrees. However, the image captured by the traditional fish-eye lens has serious distortion. Many traditional image fusion algorithms are not suitable for this kind of images. In order to handle this problem, we propose a method based on multi-band image blending to realize the real-time stitching of six-way fish-eye video, and finally generates output video. The principle of the stitching system is to decompose the video into frames. After image alignment, multi-band fusion method is used to fuse the high-frequency features, which makes the degree of fusion in the image area smaller and avoids duplication in the mosaic area. Different from the convolution pyramid stitching algorithm which does not depend too much on the overlap area, the proposed algorithm retains more image details and reduces the distortion. Experimental results in terms of both subjective and objective quality show that the system can stitch panoramic video efficiently, and the output image has no stitching artifact in the overlapping area.
Magnetic resonance imaging (MRI) is a revolutionary tool in medical imaging, which plays an important role in clinical diagnosis. Compressive sensing (CS) has shown great potential in significantly reducing the acquisition time of MRI scanning. However, how to improve the reconstruction quality with limited k-space data is still a challenge. MRI images are featured with large area of smooth regions, sharp edges and rich textures. Motivated by these facts, we propose a nonlocal autoregressive model (NAM) for CS MRI reconstruction. Nonlocal similarity between image patches is exploited as a regularization term to constrain the nonlocal feature in MRI images, which is very helpful in preserving edge sharpness. While an autoregressive regularization term is employed to describe the linear correlation between neighboring pixels, which preserves more spatial details. Different from previous work, we reconstruct an MRI image patch utilizing correlations both among patches and among neighboring pixels. Extensive experimental results demonstrate that our method outperforms mainstream methods in MRI reconstruction in terms of both subjective quality and objective quality.
Block truncation coding (BTC) is a fast image compression technique applied in spatial domain. Traditional BTC and its variants mainly focus on reducing computational complexity for low bit rate compression, at the cost of lower quality of decoded images, especially for images with rich texture. To solve this problem, in this paper, a quadtree-based block truncation coding algorithm combined with adaptive bit plane transmission is proposed. First, the direction of edge in each block is detected using Sobel operator. For the block with minimal size, adaptive bit plane is utilized to optimize the BTC, which depends on its MSE loss encoded by absolute moment block truncation coding (AMBTC). Extensive experimental results show that our method gains 0.85 dB PSNR on average compare to some other state-of-the-art BTC variants. So it is desirable for real time image compression applications.
In order to achieve the simulation of elaborate stroke trajectories in Chinese calligraphy, this paper puts forward the innovative researching on writing momentum in the field of non-photorealistic rendering in the first time. Through the analysis of using pen in Chinese calligraphy, the writing momentum is divided into three parts: the center, the side and the back of writing brush by the judgment of the angle of brush holder. We design an algorithm for dynamic outputting writing rendering based on brush model. According to monitoring parameters such as the direction, position and normalized pressure of using pen, we calculate parameters like the footprint direction, the shape, size and nib bending after writing. The algorithm can also judge the dynamic writing trend of stroke trajectories, even automatic generate stroke trajectories by the algorithm forecasted. We achieve a more delicate rendering of Chinese calligraphy to enhance the user's operating results. And we finish the unique writing effect separated the Chinese calligraphy form other general writing results, which greatly enhances the Chinese calligraphy simulation. So that people who lack of writing skills can easily draw a beautiful charm font.
The goal of sign language recognition (SLR) is to translate the sign language into text, and provide a convenient tool for the communication between the deaf-mute and the ordinary. In this paper, we formulate an appropriate model based on convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) network, in order to accomplish the continuous recognition work. With the strong ability of CNN, the information of pictures captured from Chinese sign language (CSL) videos can be learned and transformed into vector. Since the video can be regarded as an ordered sequence of frames, LSTM model is employed to connect with the fully-connected layer of CNN. As a recurrent neural network (RNN), it is suitable for sequence learning tasks with the capability of recognizing patterns defined by temporal distance. Compared with traditional RNN, LSTM has performed better on storing and accessing information. We evaluate this method on our self-built dataset including 40 daily vocabularies. The experimental results show that the recognition method with CNN-LSTM can achieve a high recognition rate with small training sets, which will meet the needs of real-time SLR system.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.