In recent years, deep learning has driven a series of major advances across machine learning. As a result, vision and language systems have started to be broadly adapted in everyday applications. However, as these models are deployed in the real world, and as we look forward to active agents over the decades to come, new problems arise and new capabilities will be necessary to solve them. In this dissertation, we sketch out two broad views: the 'input-output' view of machine learning, and the 'adaptive-active' view. In this dissertation, we focus on several key questions in the adaptive view: 1) How should agents adapt/update themselves to deal with new percepts or a changing world? 2) How should agents act in the world in order to obtain the percepts they need? 3) How should agents communicate about percepts with each other in order to communicate necessary information?