Agents that can execute natural language instructions have many applications. For example, an assistive house robot that can follow instructions will reduce the time spent on doing household chores. Natural language provides a convenient medium for users to express a wide variety of objectives for these agents. However, to achieve this goal the agent must understand the meaning of natural language instruction, reason about its context, and take appropriate actions. In this thesis, we will introduce new instruction following tasks along with new approaches. The presented approach focuses on designing scalable and interpretable agents that can follow complex natural language instructions. We also introduce an integrated learning framework for instruction following that contains an implementation of several tasks and approaches.