A Novel Method for Semi-automatic 2D to 3D Video Conversion
Chenglei Wu, Guihua Er, Xudong Xie, Tao Li, Xun Cao, Qionghai Dai, IEEE senior member
Broadband Networks & Digital Media Laboratory, Department of Automation, Tsinghua University
wucl07@mails.tsinghua.edu.cn
ABSTRACT
In this paper, we present a novel semi-automatic method for converting monoscopic video to stereoscopic video. An efficient interactive image cutout tool is first used to segment the object-of-interest in the key frames. Then, we assign the initial depth information to the segmented objects. These objects are tracked in the whole video sequence through a bi-directional KLT (Kanade-Lucas-Tomashi) algorithm. In addition, depth interpolation is employed to produce the depth information in the non-key frames. Finally, stereoscopic video is synthesized in consideration of different 3D display types. The experimental results show that our method can fulfill 2D to 3D video conversion both reliably and efficiently.
Index Terms—three-dimensional displays, image segmentation, stereo vision
1. INTRODUCTION
Three-dimensional (3D) video provides a dramatic enhancement in the viewing experience compared with its two-dimensional (2D) counterparts. Due to the huge amount of existing 2D video in many databases, converting 2D video to 3D video is of considerable practical interest not only to academic researchers but also to 3D TV industrialists.
Different approaches have been exploited in the past decades to generate the stereoscopic video. These methods can be generally divided into two categories according to whether depth maps are computed. Depth-based approaches are devoted to estimate the structure of the scene from the original 2D video, which remains a computer vision difficulty [1-6]. The other scheme is to synthesize the binocular parallax in stereoscopic video directly from monocular video without computing the scene depth information [7-9]. This kind of depth-free methods allow to convert 2D images into binocular 3D images by selecting images which could be a stereo pair of image among the original sequential images [7]. This idea is improved to get the suited right image to the left of stereo frame pair by
This work was supported in part by The Distinguished Young Scholars of NSFC, No.60525111 and in part by the 863 Program, 2007AA01Z332
performing a planar transformation [8]. [9] formulates the stereo synthesis as an optimization problem via introducing a cost function, which is used to select the most suitable frames in the input video for generating the stereoscopic video frames. However, these methods require the captured scene or objects remain almost stationary and the motion of camera must introduce a horizontal parallax, which restricts the practical implementation to certain scenario.
2D to 3D video conversion by generating a depth map from the original 2D image sequence has been widely studied in an active computer vision area called Structure-From-Motion (SFM). Correspondences between subsequent frames (or similar views of the same scene) are used to recover camera parameters and determine the depth [2]. Using the extended Kalman filter, camera motion is estimated in rotation and translation as well as the structure of scene is estimated [3][4]. For this approach, it is also required that the 3D scene must be predominately static. Another scheme is to calculate the depth maps through analyzing the motion of objects, which assumes that the motion of all objects is proportional to their distance from the camera [5][6]. This assumption only holds in a relatively small proportion of footage encountered. The state-of-the-art methods are mainly automatic 2D to 3D video conversion, while the intrinsic illness of the conversion problem prevents these methods from producing satisfactory results for arbitrary situation.
Since the depth map can be highly compressed and its resolution may be lower than the associated 2D image, these characteristics prompt us to implement the 2D to 3D conversion though depth-based scheme. The bottleneck in the 2D to 3D video conversion lies in the estimation of depth maps from the monoscopic sequence, a mathematically ill-posed problem which admits an infinite number of solutions since true depth information of the scene is unknown. As a result, we integrate user’s interactive operation into our conversion method to gain more information about the scene structure. In this paper, a novel framework for interactive 2D to 3D video conversion is proposed. Key frames are selected for user’s interactive object selection and depth assignment. To obtain the objects’ depth information in non-key frames, a modified algorithm for object tracking is employed, which utilizes the bi-directional tracking and active contour model called snake [10] to track the objects’ contour throughout the video
978-1-4244-1755-1/08/$25.00 ©2008 IEEE
3DTV-CON'08, May 28-30, 2008, Istanbul, Turkey
We propose a 3D face reconstruction method for ...I. INTRODUCTION Automatic face recognition has been... are arranged in a semi-circle around the ...
A possible method is to take the same scene again with a stereo-camera ...matching for automatic 2d to 3d video conversion, Visual Media Production, ...
a novel method for recovering consistent depth ...Our automatic depth recovery is evaluated using a...A VIDEO SEQUENCE 975 constructed on sparse 3D ...
Data-driven 3D video methods synthesize novel ... our automatic performance reconstruction pipeline ...2008. Automatic conversion of mesh animations into...
M. Burton, 100% Accuracy in Automatic Face ...2D / 3D Modeling ?Re (De)-lighting ?Pose ...(LGBPHS): A novel non-statistical model for ...
A novel strategy for integrated utilization of ... invariably maintained by automatic addition of 10...Conversion of rice straw to sugars by dilute-...
leads to a strong demand for automatic processing methods for their output....Speci?cally, a novel neural network mapping method is proposed to use a ...
A method for numerical c... 暂无评价 6页 2... Semi-Automatic Generatio... 暂无评价 9页 免费 ...Ship surface representation and 3D ship design[D]...
A 6-Speed Automatic Transmission Plant Dynamics ...The modeling method, model validation, and ...for solenoid currents to clutch pressures conversion...
A Method for Automatic POS Guessing of Chinese Unknown Words_英语学习_外语...They also attempted to apply the model to semi-supervised learning, and ...
热门文档