theses proposal
Plenoptic Modelling and Analysis
- Guarantor
-
Andrea Fusiello
- Study courses
-
PhD in Computer Science (last activated in 2013)
Description
The plenoptic function is the 5-dimensional function representing the intensity of the light observed from every position and direction in 3-D space. If the plenoptic function is constructed for a single point in space then its dimensionality is reduced from 5 to 2. This is the principle used in environment mapping where the view of the environment from a fixed position is represented by a 2-D texture map, and is analogous to the construction of panoramas or spherical mosaics. If time is added, one obtains a 3-D function describing an image sequence, also known as a spatiotemporal image volume. Nowadays we see an increasing interest in the convergence of Computer Vision and Computer Graphics. One of the most promising topic is Image-Based Modeling and Rendering (IBMR). While the traditional geometry-based system use a 3-D model, in the image-based approach, the world is modelled by a collection of example images and these are used to generate novel images representing the scenes appearance at arbitrary points in the world. This greatly simplifies the modelling of real scenes as only a number of example images need to be acquired. A second advantage of the image-based approach is that the complexity of rendering the scene is decoupled from the complexity of the scene. This is particularly important for interactive applications where fast rendering times are essential. IBMR can be cast as reconstruction of the plenoptic function from a set of examples images. Once the plenoptic function has been reconstructed it is straightforward to generate images by indexing the appropriate light rays. Emerging multimedia applications and services require efficient and flexible coding (MPEG- 4) and description (MPEG-7) of visual information. Object-based representations are particularly suited to this purpose, since they aim to describe a dynamic scene in terms of its constituents instead of low-level visual primitives. Moving object segmentation plays a central role in constructing such a representation. Typically, image sequence analysis operates on two frames only, however in the last decade a fundamentally different way of approaching image sequence analysis has been proposed, in which an intensity image sequence is considered a three-dimensional scalar field. Similarly, a color image sequence is a three-dimensional vector field. Both of these are called spatiotemporal image volumes. In spite of this promising idea, the full potential of this approach has not been realized, nor its relationship with the plenoptic function.
Sketch of research Topics:
* IBMR using Lumigraph approach.
* View synthesis with plane+parallax from user specified point of views.
* Architectural modelling from images for virtual/augmented reality applications.
* Photogrammetric scene modelling from images for forensic applications.
* Video sequence analysis (motion segmentation) exploiting the spatiotemporal image volumes representation. 3-D mathematical morphology tools will be investigated.
