Techniques for increasing reliability and scalability of live streaming systems

Abstract

Media content distribution is a widely useful tool that has seen increasing popularity driven by the rise of new technologies. File sharing constitutes one popular style of content distribution, whereby users are interested in downloading all data as fast as possible. File-sharing systems are efficient in distributing data that is fully available in advance and when there are no strict time or ordering constraints on the reception of data by interested nodes. This dissertation focuses on a second style of content distribution, in which data must be delivered at a constant rate, with minimal latency from the original time of distribution by the sender. Live streaming systems have become quite popular in recent years, and in places like China, are now widely used for broadcasting television channels to several thousands of users.

The peer-to-peer paradigm (P2P) allows live streaming systems to scale on the number of users and to be robust in the face of simple failures. By employing interested nodes' upload capacities to help upload data to other nodes, the costs at the sender are minimized and the system is able to scale to very large numbers of receivers. Unfortunately, the reliance on untrusted nodes' resources also leads to security problems and lack of guarantees, leaving such systems vulnerable to attacks that may impede nodes from receiving data in which they are interested. In particular, live streaming protocols deserve special attention since their time sensitive nature makes them more susceptible to the packet loss rates induced by attacks.

In this dissertation, I propose and evaluate techniques that enhance the scalability and resilience of live streaming systems. First, I investigate the vulnerabilities of previously proposed tree-based and mesh-based dissemination systems and present SecureStream, a P2P live streaming system built to tolerate malicious behavior at the end level. I present the main components of SecureStream and present results that demonstrate its resilience in the face of a limited class of attacks. Next, I consider the effect of selfish behavior on the distribution of data and present a practical auditing approach designed to encourage fairness in P2P streaming. I demonstrate the effectiveness of the proposed approaches through experiments using a custom built event-driven simulator and when appropriate, with an implementation of the system evaluated over the Emulab testbed.

I also address the problem of heterogeneous upload bandwidths amongst users, and propose and evaluate a hierarchical approach that allows dissemination of live streaming data in such scenarios. The approach relies on filtering the contents being disseminated, and on providing higher quality data to nodes with higher upload rates. One of the components necessary for the proposed approach is a tool that allows nodes to collect statistics about the overall upload bandwidth distribution across the system. Motivated by this requirement, I finally present NightWatch, a tool that allows nodes to estimate distributions of values held by other nodes (e.g. their upload bandwidth) in a decentralized and inexpensive manner. Such a tool has broader applicability, being also useful for monitoring purposes in large-scale distributed systems.