Towards Scalable and Optimal Oblivious Reconfigurable Networks
Datacenter network demands show explosive growth, doubling nearly every year. Unfortunately, datacenter networks are built primarily using packet switches, which do not scale as quickly as these demands and are expected to scale even slower in the future. Nanosecond-scale optical circuit switches represent a potential alternative: unlike packet switches, they are not limited by semiconductor scaling trends, and unlike previous optical circuit switches, they are fast enough to support all datacenter traffic types, including short flows. To be used to their full potential, however, these switches will require novel network designs. This dissertation examines how to build datacenter-scale networks using exclusively nanosecond-scale optical circuit switches. We identify the Oblivious Reconfigurable Network (ORN) design paradigm, which is designed to use the capabilities of these switches. We develop Shale, the first ORN to achieve a tunable tradeoff between latency scalability and throughput. We also show how to compose multiple tunings to support multiple traffic classes, a common feature of datacenter network traffic. To enable Shale, we develop a novel congestion control algorithm tailored to Shale’s unique environment, which we extend to address node and link failures. Finally, we implement a Field-Programmable Gate Array (FPGA)-based hardware prototype for a Shale end-host. Our designs show that Shale can achieve orders of magnitude better latency and hardware resource requirements than previous ORN designs. Additionally, we investigate the fundamental performance limits of ORNs, and prove that ORNs must grapple with an inherent tradeoff between latency and throughput. The tradeoffs achieved by Shale match this fundamental tradeoff up to a constant factor, meaning that every tuning of Shale is Pareto optimal among ORNs. Together, Shale and our exploration of the fundamental limits of ORNs represent important steps towards scalable and optimal ORNs.