The viscosity of colloidal suspensions varies with shear rate, an important effect encountered in many natural and industrial processes. Although this non-Newtonian behavior is believed to arise from the arrangement of suspended particles and their mutual interactions, microscopic particle dynamics are difficult to measure. By combining fast confocal microscopy with simultaneous force measurements, we systematically investigate a suspension's structure as it transitions through regimes of different flow signatures. Our measurements of the microscopic single-particle dynamics show that shear thinning results from the decreased relative contribution of entropic forces and that shear thickening arises from particle clustering induced by hydrodynamic lubrication forces. This combination of techniques illustrates an approach that complements current methods for determining the microscopic origins of non-Newtonian flow behavior in complex fluids.