Over the last few decades, recommender systems have become important in affecting people's decision making in terms of what content to watch, what places to visit, or what products to purchase. Many efforts have been made to build more accurate and diverse recommendation algorithms. In the meanwhile, new challenges such as the increasing concerns for user data privacy and fairness of user experience have caught massive public attention, calling for novel algorithm and user-interaction design to satisfy those emerging social-technical needs. In this thesis, we develop user-aware recommender systems for individuals and groups. On the individual level, we investigate how the quantity and quality of user feedback data affect recommendations. We start by exploring how collaborative filtering algorithms would be affected by user-controlled filtering of feedback data, in a scenario where each user has different temporal sharing preferences. We find that the recommendation performance may not be negatively affected even with less user data, allowing certain flexibility for users to satisfy their privacy needs. To address the limitations of existing feedback data, we demonstrate the effectiveness of post-click feedback such as click-skip or click-complete in training and evaluating a recommender. We propose a probabilistic framework to fuse click and post-click feedback and show how our framework can be applied to improve pointwise and pairwise recommendation models. To reduce the uncertainties in post-click feedback, we collect a music rating dataset with 500K binary choices of likes and dislikes, and demonstrate the value of such binary feedback for improving music recommendations. We release the dataset to the public with the goal of enabling researchers and practitioners to explore the limitations of existing recommendation datasets and to inspire better design of user feedback elicitations. On the group level, we focus on the experience of sub-populations and multiple stakeholders. We observe that popular recommendation algorithms suffer from unbalanced performance among user subgroups. Moreover, we find that the initial gap is further enlarged after multiple rounds of recommendations. To mitigate such amplification effects, we take a first step in exploring the Distributionally Robust Optimization framework to optimize for worst-case performance in recommendations. We propose a simple-yet-effective streaming optimization improvement to reduce loss variances. Our results on real-word recommendation datasets suggest the promise of this framework by improving the worst-case and overall performance at the same time. In addition to end-user experience, we take a quantitative approach to understand the experience of different stakeholders. In a case study with a music recommender system, we evaluate the performance of classic recommendation algorithms on three important stakeholders: consumers, well-known artists and lesser-known artists. We show that a matrix factorization algorithm trained on binary-choice feedback of likes and dislikes performs significantly better compared to one trained on implicit feedback for all three stakeholders. At the end of the thesis, we discuss future directions and open research questions where recommender systems aspire to satisfy the value and needs for their stakeholders and the society.