My research focus is on developing algorithms and statistical theories of sequential decision making on the intersection of reinforcement learning (RL) and causal inference. RL is concerned with the ways agents learn to make sequential decisions in unknown environments. It has been one of the most vibrant research frontiers in machine learning over the last few years. We have empirical success in a variety of applications, especially for games such as AlphaGo (Silver et al., 2016). Despite its popularity, the real-world application of RL in fields such as biomedicine and social science is still limited. This is because these real-world applications do not have good simulators, and experimentation is often expensive and risky (e.g., running clinical trials, deploying new marketing strategies in companies) unlike for games. Although running new experiments can be difficult, fortunately, in an era of big data, we often have access to massive historical datasets such as web-logged data and large electronic health records. This motivated me to find ways to use offline data in a statistically efficient manner, which is a central topic in the subfield of offline RL and causal machine learning. However, there is a certain limitation in offline RL when the quality of the offline data is poor. In this scenario, we want to find the best policy by adaptively collecting data. This motivated me to find ways to collect the data and search for the best policy, which is a central topic in online RL. Since experiments are often costly, it again needs to be performed in a statistically efficient way. Hence, building statistically efficient RL algorithms in both offline and online settings is the key to bringing RL to a variety of real-world applications.