United States Evaluation Policy: A Theoretical Taxonomy

Abstract

Efforts are currently underway in the US federal context to improve and strengthen evaluation practice and increase the use of evaluation results to inform policies and programs. However, these efforts remain unrealized, due partly to the lack of a comprehensive theoretical framework that views evaluation and related organizational processes and institutions as part of a larger system. Early intuitive theoretical taxonomies of evaluation policy suffer from the lack of connection to specific examples and instances, and are missing clear classification criteria that would allow practical application. To generate a grounded taxonomy of evaluation policy, this study surveyed members of the American Evaluation Association in 2009, asking them to generate examples of evaluation policy, and then to sort and rate these suggested policies. Results are analyzed using the concept mapping method of Trochim (1989), which first translates aggregate sorting decisions into conceptual "distances" on a two-dimensional dot map, then uses hierarchical cluster analysis to generate groupings of ideas. These groupings become the foundation for categories in a theoretical taxonomy. Findings reveal several different dimensions by which participants grouped evaluation policies, including the dimensions of "value" and "policy mechanism." A values-by-mechanisms taxonomy and instructions for its use in an evaluation policy inventory process are proposed. iii