Novel view synthesis with residual error feedback for FTV

Hisayoshi Furihata; Tomohiro Yendo; Mehrdad Panahpour Tehrani; Toshiaki Fujii; Masayuki Tanimoto

doi:10.1117/12.839110

18 February 2010 Novel view synthesis with residual error feedback for FTV

Hisayoshi Furihata, Tomohiro Yendo, Mehrdad Panahpour Tehrani, Toshiaki Fujii, Masayuki Tanimoto

Proceedings Volume 7524, Stereoscopic Displays and Applications XXI; 75240K (2010) https://doi.org/10.1117/12.839110
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States

Abstract

The availability of multi-view images of a scene makes possible new and exciting applications, including Free-viewpoint TV (FTV). FTV allows us to change viewpoint freely in a 3D world, where the virtual viewpoint images are synthesized by Depth-Image-Based Rendering (DIBR). In this paper, we propose a new method of DIBR using multi-view images acquired in a linear camera arrangement. The proposed method improves virtual viewpoint images by predicting the residual errors. For virtual viewpoint image synthesis, it is necessary to estimate the depth maps with multi-view images. Some algorithms to estimate depth map were proposed, but it is difficult to estimate accurate depth map. As a result, rendered virtual viewpoint images have some errors due to the depth errors. Therefore, our proposed method takes into account those depth errors and improves the quality of the rendered virtual viewpoint images. In the proposed method, the virtual images of each camera position are generated using the real images from each other camera. Then, the residual errors can be calculated between the generated images and the real images acquired by the actual cameras. The residual errors are processed and fed back to predict the residual errors that can be happened to virtual viewpoint images generated by conventional method. In the experiments, PSNR could be improved for few decibels compared with the conventional method.

Citation Download Citation

Hisayoshi Furihata, Tomohiro Yendo, Mehrdad Panahpour Tehrani, Toshiaki Fujii, and Masayuki Tanimoto "Novel view synthesis with residual error feedback for FTV", Proc. SPIE 7524, Stereoscopic Displays and Applications XXI, 75240K (18 February 2010); https://doi.org/10.1117/12.839110

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available