Video Object Coding and Relighting for Image Base Representation

This dissertation, "Video Object Coding and Relighting for Image Base Representation" by Kin-lok, Chan, 陳健樂, was obtained from The University of Hong Kong (Pokfulam, Hong Kong) and is being sold pursuant to Creative Commons: Attribution 3.0 Hong Kong License. The content of this dissertation has not been altered in any way. We have altered the formatting in order to facilitate the ease of printing and reading of the dissertation. All rights not granted by the above license are retained by the author. Abstract: Abstract of thesis entitled Video Object Coding and Relighting for Image Base Representation submitted by Chan Kin Lok for the degree of Master of Philosophy at the University of Hong Kong August 2004 Most proposed image base representations and their relighting algorithms focus on static environments or scenes, mainly due to the logistic difficulties in capturing and transmitting dynamic representations, which required large amounts of data. The objective of this study was to investigate a compact solution for interactive dynamic image base representations. The proposed solution was to incorporate dynamic video objects into image base representations with interactive changes in lighting. Since the proposed solution generates dynamic scenes instead of recording, the storage requirement is significantly reduced. Experiments are performed to examine the feasibility of the proposed solution. Without loss of generality, experiments focus on the image base representation known as panoramic video. Three problems are addressed in the proposed solution: (i) interactive rendering of video objects in panoramic videos; (ii) interactive relighting of these objects and videos; (iii) compression of these objects and videos. Several algorithms are proposed to resolve these problems. In the proposed rendering algorithm, video objects are projected onto panoramic video using backward projections, after defining the trajectory and paths of these objects inside panoramic videos. This approach allows flexible video objects to be rendered in any location and orientation inside panoramic videos. In order to further enhance the user's experience, interactive relighting of these objects and videos is desirable. Many existing relighting algorithms for image base representations rely on densely sampled images to perform relighting, and inevitably require a large amount of storage for sample images. To overcome this problem, a new relighting algorithm is proposed. In this approach, depth maps and a reflection model are used to approximate relighting results so that only depth maps and colour information on these objects and videos need to be stored. As a result, data storage is greatly reduced and data acquisition is considerably simplified. To further reduce the required storage size, depth maps and colour information are compressed. Since depth maps, video objects and panoramic videos are sequences of continuous tone correlated images, redundancies exist inside or between images. Various compression algorithms are proposed to fulfill different compression requirements for depth maps, video objects and panoramic videos. Experiments using synthetic objects and videos are conducted to evaluate the proposed rendering, relighting and compression algorithms. The results demonstrate dynamic video objects can be successfully incorporated into panoramic videos with interactive relighting, where shading and video objects shadows are rendered consistently. The compression results also show that a satisfactory image quality can be preserved even under a considerable compression ratio. (Abstract of exactly 408 words) DOI: 10.5353/th_b3022196 Subjects: Video compression Lighting Algorithms