Optimal transport (OT) distances provide a way to compare probability measures with rich geometric and topological properties that align well with human perception in many areas of machine learning. However, OT distances suffer from computational and statistical scalability issues to high dimensions, which motivated the study of regularized OT methods like slicing, smoothing, and entropic penalty. In this thesis, I will discuss several applications of regularized OT distances towards problems in non-parametric inference and generative modelling, and how regularization helps address some issues with vanilla OT. I will also discuss some aspects of dimension adaptation of regularized OT distances, and the unregularized semi-discrete setting where the statistical convergence issue can also be resolved.