There is intense fundamental and technological interest in perovskite oxides, a large class of ceramic materials that exhibit an unprecedented diversity of physical properties. Some perovskites exhibit ferroelectricity, the existence of a field-switchable electric polarization. Other perovskites exhibit rotations of the oxygen octahedra, which buckles bonds and leads to rich phase diagrams of magnetic and electronic order. This thesis investigates perovskites in which ferroelectricity and octahedral rotations appear together and are strongly coupled. We use first-principles methods and simple physical models to elucidate trends across multiple materials. The first part of this thesis introduces a new model for the interplay of ferroelectricity and octahedral rotations based on ferri-electricity. The second part of this thesis demonstrates for the first time perovskites that exhibit coupled ferroelectricity and octahedral rotations have electrical properties that are not possible in perovskites that exhibit ferroelectricity alone. The final part of this thesis is the first theoretical study into ferroelectric switching that takes into account the variety of structural domains that arise due to the octahedral rotations. These findings contribute to understanding how novel properties can emerge in materials with strongly coupled degrees of freedom.