At the intersection of several pure and applied mathematical disciplines, optimal transport (OT) has been drawing surprising connections across geometry, analysis and statistics, while also offering powerful computational methods in data science and machine learning. The increasing interest in recent years in heterogeneous data has, however, not been quite compatible with OT, and requires systematic treatment of a closely related but significantly harder object called the Gromov-Wasserstein (GW) alignment. The GW framework inherently only utilizes the intrinsic structure of data, modeled as metric measure spaces, and has seen widespread applications. Despite its compelling mathematical formulation, GW encompasses several stark structural differences, prohibiting the extension of much of classical OT studies to the GW problem, while hindering the development of better computational methods for GW. This thesis addresses these gaps, motivated by establishing the transport aspect of GW. The first contribution is a novel duality theory for GW and the first statistical estimation rate of the (2,2)-GW distance and its entropic regularized variant, with matching lower bounds. For better applicability, the second contribution of this thesis seeks statistically and computationally efficient heterogeneous methods under GW's framework. The third contribution delves deeper into drawing parallels from OT's Riemannian structure to GW, leading to novel gradient flow and dynamical formulation.