Video Registration and 3D Reconstruction for Deformable Surfaces

Our goal is to produce a 3D model of a deformable surface from a video. The video can show for instance a paper sheet or a piece of cloth being bent. Without a deformation model, monocular Structure-from-Motion is an ill-posed problem, even if the camera is internally calibrated. Our basic assumption is that the observed surface is continuous and smooth, to some extent. We first show how the video can be registered by selecting a reference frame and estimating a warp to each of the other frames. We then proceed to reconstruct the 3D surface as seen in each frame of the video, using the low-rank shape model in a hierarchical manner. Surface retexturing and full 3D augmentation examples will be shown.