Network and content adaptive streaming of layered-encoded video over the Internet

In this thesis we propose new techniques and algorithms for improving the quality of Internet video streaming applications. We formulate optimization problems and derive control policies for transmission over the current best-effort Internet. This dissertation studies adaptation techniques that jointly adapt to varying network conditions (network-adaptive techniques) and to the characteristics of the streamed video (content-adaptive techniques). These techniques are combined with layered-encoding of the video and client buffering. We evaluate their performance based on simulations with network traces (TCP connections) and real videos (MPEG-4 FGS encoded videos). We first consider the transmission of stored video over a reliable TCP-friendly connection. We compare adding/dropping layers and switching among different versions of the video; we show that the flexibility of layering cannot, in general, compensate for the bitrate overhead over non-layered encoding. Second, we focus on a new layered-encoding technique, Fine-Granularity Scalability (FGS), which has been specifically designed for streaming video. We propose a novel framework for streaming FGS-encoded videos and solve an optimization problem for a criterion that involves both image quality and quality variability during playback. Our optimization problem suggests a real-time heuristic whose performance is assessed over different TCP-friendly protocols. We show that streaming over a highly variable TCP-friendly connection, such as TCP, gives video quality results that are comparable with streaming over smoother TCP-friendly connections. We present the implementation of our rate adaptation heuristic in an MPEG-4 streaming system. Third, we consider the general framework of rate-distortion optimized streaming. We analyze rate-distortion traces of long MPEG-4 FGS encoded videos, and observe that the semantic content has significant impact on the encoded video properties. From our traces, we investigate optimal streaming at different aggregation levels (images, groups of pictures, scenes); we advocate scene-by-scene optimal adaptation, which gives good quality results with low computational complexity. Finally, we propose a unified optimization framework for transmission of layered-encoded video over lossy channels. The framework combines scheduling, error protection through Forward Error Correction (FEC) and decoder error concealment. We use results on infinite-horizon average-rewards Markov Decision Processes (MDPs) to find optimal transmission policies with low-complexity and for a wide range of quality metrics. We show that considering decoder error concealment in the scheduling and error correction optimization procedure is crucial to achieving truly optimal transmission