Reinforcement Learning (RL), the maximization of expected cumulative reward, is a framework for interactive decision-making that has produced many successes in games, recommendation systems, drug discovery, and large language models (LLMs). Despite these successes, existing algorithms can be brittle and inefficient, making RL difficult and costly to use, especially in safety-critical settings. My broad goal is to make RL agents much more efficient, robust, and practical, and in this thesis, we make progress towards this goal via the following three directions: 1. Safety and Robustness: How can we train agents to avoid unsafe behaviors while being robust to unseen or even adversarial situations? 2. Efficiency and Adaptivity: How can we design algorithms that use available data to the fullest and that can adaptively learn faster in realistic (non-adversarial) environments? 3. Steerable and Transferable RL: How can we use vast knowledge from related tasks to reduce the cost of exploration and representation learning, and to learn a steerable multi-task agent? Each aim is developed in a dedicated chapter, beginning with foundational theory and algorithms, and culminating in real-world applications that demonstrate practical impact. Together, they chart a path toward RL that is safer, more efficient, and more broadly usable.