Object-Based Coding and Watermarking for Image-Based Rendering

This dissertation, "Object-based Coding and Watermarking for Image-based Rendering" by Xinzhi, Yao, 姚欣志, was obtained from The University of Hong Kong (Pokfulam, Hong Kong) and is being sold pursuant to Creative Commons: Attribution 3.0 Hong Kong License. The content of this dissertation has not been altered in any way. We have altered the formatting in order to facilitate the ease of printing and reading of the dissertation. All rights not granted by the above license are retained by the author. Abstract: Image-based rendering (IBR) has emerged as an important technique in virtual reality, digital museums, interactive visualization in multi-view TVs and many other rapidly developing areas in the information and communication technology industry. IBR utilizes densely sampled two-dimensional images to generate novel views needed at different viewpoints to describe the three-dimensional scene. IBR representations (image-based representations) usually involve large data sizes; thus their efficient compression is vital for IBR's practical use. As IBR becomes more widely applied in academia and industry, protecting various image-based representations becomes increasingly important to ensure its proper use and the author's intellectual property. Digital watermarking is a promising way to solve this issue. Though previous research has studied compression of IBR, efficient system has not yet been fully investigated. Meanwhile, watermarking in IBR is still a new demanding area which needs effective schemes to be developed. This work focuses on object-based coding and feature-based watermarking for image-based representations. Firstly, a multi-view object coding framework is proposed for image-based representations based on the Audio Video Coding Standard of China (AVS). Object-based coding compresses the IBR data (usually multi-view images/videos) at the object level. Image-based representations are first processed using object-based approach to segment and extract different objects within the data, each with their corresponding texture, depth map and shape information. The segmented objects are then compressed with state-of-the-art AVS coding techniques and tools. AVS-based object coding has the advantage of less complexity compared with H.264/AVC, while being more efficient than standardized object coding available in MPEG-4. The proposed framework supports multi-view coding to explore the redundancy between different views of the IBR data with efficient inter-frame and inter-view coding mode. Object-based temporal scalability is also achieved based on the proposed multi-view object coding framework. Secondly, a novel two-pass rate control framework is proposed based on a non-linear exponential rate-distortion model. Convex optimization is utilized to allocate the optimal bits among different coding units at different levels. Region-of-interest is readily achieved through assigning different important factors to different objects. Rate control with object-based temporal scalability is also addressed for object-based adaptive transmission. At the same time, an analytical model-based bit-allocation approach is further proposed as a complement of convex optimization-based approach towards real-time applications. Lastly, a feature-based watermarking system for copyright protection of image-based representations is developed. The proposed scheme uses scale invariant feature transform to extract robust feature points and formulate corresponding feature patches centered at the reference points in all the IBR views for watermark embedding. Discrete Fourier Transform coefficients of each patch are modified to embed a circular symmetric 2-D watermark pattern generated with a secret key. The watermark is synchronized by a hierarchical non-rigid image registration method to resist the effect of IBR and various geometrical attacks. Correlation-based detection is applied on each synchronized patch to determ