We study the ability of bipedal walking robots both to avoid falling down and to reach a specific goal, such as standing still or moving along a desired trajectory. The main questions are: When - in what states and with what controls - is it possible for the robot to avoid a fall or to reach a given goal? When reaching the goal is possible, how fast (in how many steps) can it be done? For various meanings of the word 'optimal', what are optimal ways of getting to the goal? Our primary approach uses two simple 2D models of walking: the Inverted Pendulum (IP) and Linear Inverted Pendulum (LIP). Both models have a point mass at the hip and two massless legs: rigid legs in the IP, extensible in the LIP model. The hip of the LIP is constrained to move at a fixed height above the ground. Both models have two controls per step: step length and impulsive push-off force along the stance leg just before heel-strike in the IP, and step length and stepping time in the LIP model. For each model we numerically compute the extended n-step viable regions: all combinations of states and controls that let the robot take at least n steps and not fall. We also compute the extended n-step controllable regions: all states and controls that let the robot reach a given steady-state-walking goal by taking n steps or fewer. Using these models we justify two claims: (i) In most cases and for most bipeds, when the biped can reach a given goal at all, it can do so within two steps. This result is consistent with some human walking-balance and visual- guidance experiments. (ii) The ability to reach a specific steady-state target is almost equivalent to being able to avoid totally falling down. The controllable regions of the IP model also help us understand the trade-offs between different controllers. Such trade-offs are in robustness, convergence to the nominal motion, and energy efficiency. Based on things learned, we design a walking controller for a real robot Cornell Ranger, which is nearly maximally robust, near optimal in some regards, and is simple in structure.