Machine learning has recently achieved significant milestones, resulting in potent real-world applications such as chatbots, autonomous driving, and protein structure discovery. Despite these advancements, the question of trust in machine learning models remains paramount. This thesis addresses two primary concerns regarding the trustworthiness of machine learning models: privacy and robustness. Privacy concerns arise when training data, potentially encompassing sensitive personal or medical information, is utilized. Previous studies validate the risk of such information leakage through machine learning models trained on this data. Robustness represents another challenge, as the distributions of training and testing data are not always aligned, leading to unexpected performance outcomes. The difficulty lies in enhancing the robustness of machine learning models against such data distribution shifts. This thesis delves into the problems of privacy and robustness in machine learning models. In terms of privacy, the discussion revolves around three focal points: (i) vulnerabilities exposed in federated learning; and (ii) the semantic protection under label differential privacy; (iii) privacy-preserving training for image generative models leveraging public data. In terms of robustness, the focus is on (i) enhancing the robustness of paper-reviewer assignments against bid manipulation attacks and (ii) enabling machine learning models to adapt to label distribution shifts in an online setting. The goal of these discussions is to stimulate further research in trustworthy machine learning, thereby fostering the responsible development of machine learning in real-world applications.