Machine-learning research is progressing rapidly thanks to advances in model architectures, optimization, training algorithms, and scaling. There are many ingredients that allow this rapid progress. One is the ability to quickly build on prior papers, which depends on the clarity of the papers, and the availability of accompanying research code. Another essential ingredient is the correct evaluation of algorithms, which ensures that the papers accurately portray their algorithm's performance. This is important because researchers think of new ideas under the assumption that prior works contain accurate information. If papers do not evaluate algorithms correctly, then the inaccurate results could lead future researchers astray. Some machine learning sub-fields are studied very actively. With so many critical eyes, it is unlikely for incorrect evaluation methodologies to propagate through the literature. However, some other sub-fields attract relatively little scrutiny. As a result, various flawed evaluation methodologies can go unchallenged for years. This leads to distortions in experiment results, which misleads researchers and hinders progress. In this dissertation, we investigate two machine-learning sub-fields: metric learning and unsupervised domain adaptation. We reveal the flaws in the evaluation methodologies of both fields, and present improvements.