Scaling Searchable and Transactional Storage Systems

Abstract

Data is the lifeblood of modern computing and the systems that store it have taken a prominent place in the infrastructure of practically every modern startup, business, or research application. Not-so-recent trends in distributed storage systems have removed features---such as secondary attribute search or transactions---that applications used to take for granted. These missing features must be reimplemented at the application level, or the application must be carefully constructed to work around their absence. This thesis explores work on four systems that represent advances in reversing this trend. First, it looks at HyperDex, a system which provides efficient secondary attribute search. Second, it presents two transactional storage systems, Warp and Consus. Warp targets a single data center environment while Consus targets a geo-replicated deployment and the differences in their design reflect these two considerations. Finally, this thesis presents the Warp Transactional Filesystem that shows a positive example of how the transactional properties of Warp can be extended to provide application-level transactional guarantees. Finally, the thesis looks at the broader impact of these systems and how the evolution of the systems could be used to inform the development of future distributed systems.