Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell University Graduate School
  3. Cornell Theses and Dissertations
  4. BUILDING NETWORKED SYSTEMS FOR TERABIT ETHERNET

BUILDING NETWORKED SYSTEMS FOR TERABIT ETHERNET

File(s)
Cai_cornellgrad_0058F_14532.pdf (2.38 MB)
Permanent Link(s)
https://doi.org/10.7298/nkw2-e114
https://hdl.handle.net/1813/116404
Collections
Cornell Theses and Dissertations
Author
Cai, Qizhe
Abstract

Over the last two decades, hardware in datacenters has shown diverging trends. On the one hand, access link bandwidth has increased rapidly, with datacenters now commonly supporting Terabit Ethernet (i.e., Ethernet with speeds above 100Gbps). Modern network hardware is capable of supporting microsecond-scale latency and multi-hundred-gigabit bandwidth. However, on the other hand, the slowdown of Moore’s Law and the end of Dennard scaling have resulted in total compute capacity (the number of CPU cores multiplied by per-core performance) remaining largely stagnant. As a result, network performance bottlenecks have shifted to the host network stacks, which are responsible for processing network packets to and from applications. The first contribution of this dissertation is to build an in-depth understanding the core challenges hindering existing host network stacks from fully leveraging modern network hardware. Our study reveals that the rapid increase in network link bandwidth has made data movement overheads (such as transferring data from NICs to CPUs) a bottleneck for scaling single-core performance. Typically, on the receiver side, after the NIC DMAs data to memory, the limited memory bandwidth leads to poor CPU efficiency, as CPUs must stall while waiting for data to be loaded from memory into CPU registers. We find out existing optimization techniques like DDIO, which enables NICs to directly read/write data from/to CPU caches, are unable to improve CPU efficiency. This is because the increase in bandwidth-delay products has outpaced the increase in cache sizes, leading to high cache miss rates and poor CPU efficiency. With today's network stacks, multiple cores are needed to fully exploit the capabilities of Terabit network hardware. However, our study shows that using multiple cores, while saturating the link bandwidth, leads to even worse CPU efficiency compared to the single-core case. This is because host resources like cache and access link bandwidth are contended among different cores/applications. The second contribution of this dissertation is to introduce NetChannel—a new network stack architecture that enables host network stacks to leverage network hardware without requiring application modifications. NetChannel disaggregates network stacks into multiple loosely-coupled layers, allowing each layer to scale and schedule across multiple cores independently. Using an end-to-end NetChannel realization within the Linux network stack, we demonstrate that NetChannel enables new operating points—(1) enabling a single application thread to saturate multi-hundred-gigabit access link bandwidth; (2) enabling near-linear scalability for small message processing with an increasing number of cores, independent of the number of application threads; and, (3) enabling isolation of latency-sensitive applications, allowing them to maintain $\mu$s-scale tail latency even when competing with throughput-bound applications operating at near-line rate. This thesis leaves open several interesting directions of future research: 1) improving CPU efficiency by reducing both data movement and CPU processing overheads; 2) extending NetChannel to function in more realistic scenarios, allowing us to fully realize its potential; and 3) extending the study to understand network stack overheads, not only in terms of CPU efficiency and throughput but also network latency, as achieving low latency is also crucial for applications.

Description
122 pages
Date Issued
2024-08
Keywords
Host Network Stack
•
Networking
•
Operating System
•
Terabit Ethernet
Committee Chair
Agarwal, Rachit
Committee Member
Shmoys, David
Sampson, Adrian
Degree Discipline
Computer Science
Degree Name
Ph. D., Computer Science
Degree Level
Doctor of Philosophy
Type
dissertation or thesis
Link(s) to Catalog Record
https://newcatalog.library.cornell.edu/catalog/16612018

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance