MITIGATING CRITICAL-PATH COORDINATION COSTS IN LARGE-SCALE DISTRIBUTED SYSTEMS
Large-scale, geo-replicated distributed systems are prevalent in modern cloud-based applications. A single cloud service typically comprises multiple small components—commonly known as microservices—and requires a variety of system-level operations, in addition to the application logic itself, to function reliably and present the illusion of a single, unified application to users and developers. These operations provide essential system capabilities such as fault tolerance, strong consistency, and coordination among distributed components, ensuring that system behaviors remain predictable and correct. However, such mechanisms unavoidably lie in the critical path of user-facing operations, substantially impacting overall performance—particularly end-to-end latency. This dissertation presents two complementary approaches that remove or mitigate the impact of operations that lie on the critical path, thereby improving latency and overall system performance.Off-Ramp Total Order predicts the future and pre-orders event execution, removing the need for traditional coordination in the critical path to achieve consensus on execution order. Dual-Outcome Speculation peeks into the future and enumerates all possible outcomes in cross-component coordination, utilizing otherwise idle computational resources—typically waiting on coordination—to perform speculative work and accelerate the response of subsequent operations once the coordination outcome is finalized. I demonstrate these approaches through the design of two systems: Ziplog and Multiverse.Ziplog implements a shared log abstraction that stores records with their assigned unique sequence numbers, establishing a single ground truth for execution order. It leverages off-ramp total ordering to pre-order records before they are submitted, disseminating these orders to all replicas in advance and thereby eliminating in-critical-path coordination. Multiverse targets multi-partition transactions where cross-partition coordination is required to ensure atomicity. It speculates on both possible outcomes—commit and abort—creating separate universes for each and executing subsequent transactions speculatively in both worlds. Beyond speculative pre-computation, Multiverse enables early identification of transactions whose outcomes are destined—that is, identical across all possible futures—allowing the system to prune redundant work and further improve overall performance.