Neurodevelopmental impairment in infancy is an established risk factor for poor cognitive and behavior outcomes later in life. However, environmental factors such as high quality caregiving could be protective against these biological risks. Longitudinal data from the National Survey of Child and Adolescent Well-Being was used to assess the relationship between difficult temperament and risk for neurodevelopmental impairment in infancy and outcome behavior problems and executive function five years later. Quality of caregiving these children received at both ages was measured. Change in caregiving quality was tested as a moderator in difficult temperament predicting behavior problems, and risk for neurodevelopmental impairment predicting executive function. It was hypothesized that increases in the quality of caregiving would result in (a) fewer problem behaviors based on initial measures of difficult temperament, and (b) better than expected executive function outcomes based on initial risk for neurodevelopmental impairment. Difficult temperament was found to predict problem behaviors 5 years later. Similarly, risk for neurodevelopmental impairment in infancy was found to predict lower executive functioning 5 years later. Findings from this study could guide the development of effective interventions for children displaying neurodevelopmental impairment shortly after birth.