New Real-time View Synthesis Method Using Spatiotemporal Background Information

Real-time virtual view synthesis technique has been widely used in kinds of 3D television systems. The image distortion is its main problem for lack of reference information in image inpainting process. This paper proposes a new view synthesis method to solve this problem based on DIBR techniques. First, four common real-time hole-filling algorithms are tested and analyzed, the proposed method automatically selects different algorithms depending on the size of the hole to be filled. Then, a process called depth Erode is designed to remove the pixels around boundaries between foreground and background in the reference image. These pixels are hard to be correctly segmented and cause artifacts in synthesized views. Finally, a structure called Background Table is proposed to extract and store spatiotemporal background information. The background table is updated simultaneously and gives a quick access to background information. The usage of spatiotemporal background information increases the reference information in image inpainting process and greatly reduces inpainting distortion. Our experimental results show that the proposed method has excellent subjective and objective performance over state-of-art methods.