Evidence In Practice: A Study Of "Evidence-Based" Non-Formal Education

Abstract

The purpose of this study is to better understand how efforts to make non-formal education more "evidence-based" actually unfold in practice. I focus specifically on three cases, two of which involve mandates to implement evidence-based programs, while the third involves incorporation of evidence-based practices. The three cases are instances of a larger contextual shift towards more "scientific" approaches to education research and evaluation, part of the "era of accountability." That shift has fomented much debate in recent years, producing theoretical critiques that-while insightful-are often rhetorical and polemical, lacking a clear grounding in particular practical settings. To address that lack, this study analyzes what actually happens, in practice, when people support the implementation of evidence-based programs or engage in related efforts to make non-formal education more "evidence-based." In essence, I ask: What does that work entail? Specifically, I ask (1) How is evidence-based program and evidence-based practice work actually practiced? (2) What perspectives and assumptions about what non-formal education is are manifested through that work? and (3) What conflicts and tensions emerge through that work related to those perspectives and assumptions? Empirically, this qualitative study is based on data from in-depth interviews, observation, and document analysis. Theoretically, it is informed by critical perspectives on epistemological politics, drawing especially from the field of science and technology studies. Through my elucidation of the intricacies and contingencies involved in making nonformal education more "evidence-based," I am able to highlight divergent perspectives and assumptions about what non-formal education is and about how it should be informed by research evidence. Specifically, non-formal education is constituted by some people as an infrastructure for the dissemination of scientific information and by others as a grassroots site for knowledge sharing. Relatedly, it is alternatively perceived and performed as a program (meaning a tightly bounded, scripted curriculum) or as a set of practices. While the conflicts and tensions between these divergent perspectives may appear academic or irrelevant, I argue that they have stark implications for what non-formal education is and can be in society, concluding that the self-evident superiority of evidence-based programs must be revisited.