On rate-distortion modeling and extraction of H.264/SVC fine-granular scalable video

Fine-granular scalable (FGS) technologies in H.264/AVC-based scalable video coding (SVC) provide a flexible foundation to accommodate different network capacities. To support efficient quality extraction, it is important to obtain the rate-distortion (R-D) or Distortion-Rate (D-R) function of each individual picture or a group of pictures (GOP). In this paper, firstly, the R-D function of SVC FGS pictures is analyzed with generalized Gaussian model and the D-R curve is proved to be a concave function overall. Considering the current sub-bitplane technology, the D-R function is revisited and inferred to be linear under MSE criterion within an FGS level, which also explains why the observed D-R curve with PSNR criterion is a piece-wise convex function. Secondly, the drift issue of SVC is analyzed, and a simple and effective distortion model is proposed to estimate the reconstruction distortion with drift error. Thirdly, with the above analysis and models, a virtual GOP concept is introduced, and a new priority setting algorithm is designed to achieve the optimal R-D performance in a virtual GOP. The D-R slope of each FGS packet and the D-R function of each virtual GOP are also obtained during the process. Finally, the D-R slopes of FGS levels are used in quality layer assignment to achieve equivalent coding efficiency to the SVC test model but with significantly reduced complexity. The D-R functions of virtual GOPs are utilized to design a practical method for smooth quality reconstruction. Compared to the prior methods, the smoothed video quality is improved not only objectively but also subjectively.