Most tasks we want robots to perform require interaction with the physical world via an unknown sequence of parameterized actions; as such, they involve both a continuous geometric component and a discrete symbolic component. For example, a robot cleaning a cluttered surface must reason about how to move objects for cleaning, both over discrete decisions about the order in which to manipulate objects and continuous decisions about grasps and placement locations. Similarly, a robot cooking a meal will need to use a different sequence of discrete actions with continuous parameters (e.g., chopping, stirring, etc.) to follow a recipe, and will need to account for geometric constraints while cooking. Integrated Task and Motion Planning (TMP) is a class of holistic approaches to solving robot planning problems with intricate, interacting symbolic and geometric constraints. TMP offers a promising route to increased autonomy for many real-world robotics tasks, but its requirement for expert-crafted symbolic action models and continued need for higher performance holds back its practical applicability. This thesis describes work toward addressing these challenges, based on a new framing of TMP as multimodal motion planning guided toward subgoals corresponding to states from which a robot agent can take actions to complete a specified task. We represent these subgoals as formulae composing differentiable functions computing the distance to the nearest state at which a desired property holds. This framing enables us to build two planners which use different approaches to action selection and subgoal sampling to provide competitive performance on TMP problems while requiring less expert specification as input than alternatives. We further contribute work on automatic symbolic-geometric abstraction repair, a means of easing the specification burden for TMP by allowing initially incorrect or incomplete symbolic abstractions and modifying these abstractions to iteratively converge toward a correct model over time. This work includes an alternative formulation of differentiable-distance-function predicates that offers tight representation of a broader class of non-convex state sets, formally defines the symbolic-geometric abstraction repair problem, and provides a simple anytime bilevel optimization approach to performing abstraction repair from a handful of ``unexpected'' action execution observations. Finally, this thesis concludes with a brief overview of the current state and incipient directions of TMP research, with an eye toward the problems that remain outstanding for TMP to become practically useful outside of the lab.