Analytic Alternatives For Evaluating Human Services Interventions
Files
No Access Until
Permanent Link(s)
Collections
Other Titles
Author(s)
Abstract
Human service programs and clients are complex. Clients in publicly-funded human service interventions frequently are dealing with multiple, complex, cooccurring issues and conditions, and human service agencies are increasingly responding with comprehensive, integrated, individualized service approaches. The goal of this dissertation is to investigate the utility of alternative analytic techniques that might provide greater "representational complexity" - richness and detail of information about clients and their interactions with programs - than standard, "off-the-shelf" analytic techniques provide. Two analytic approaches that are not often used in human service evaluation studies but that might lead to higher levels of representational complexity are assessed in two separate substudies. Within each sub-study the same evaluation data is analyzed using an alternative technique and a current, standard technique. The first study, a comparison of effectiveness between two programs, draws on data from a national cross-site evaluation of substance use treatment for youth. In this study the alternative technique Maximum Change Scores (MCS - Boothroyd, Banks, Evans, Greenbaum, & Brown 2004) is tested against the standard technique for this situation, Hierarchical Linear Models (HLM - Raudenbush & Bryk, 2002; Snijders & Bosker, 2012; Singer & Willett, 2003). The second study, an examination of within-program patterns of differential effectiveness, draws on data from a case management program for homeless women. In this study, the tree-based modeling technique Classification and Regression Trees (CART - Breiman, Friedman, Olshen, & Stone, 1993) is pitted against standard linear models for examining whether different types of clients showed different patterns of change while in the program. The innovative techniques are evaluated against their standard counterparts on five dimensions: 1) comparability of results, 2) representational complexity, 3) utility to service providers, 4) validity, and 5) data requirements. In the first study, MCS produced results that were overall comparable with those from HLM. The technique worked as expected in representing clients change more flexibly, but its usefulness is limited because it is not inherently longitudinal and is less flexible than HLM. In the second study, the tree models produced results that differed substantially from the linear model results but provided higher representational complexity.
Journal / Series
Volume & Issue
Description
Sponsorship
Date Issued
Publisher
Keywords
Location
Effective Date
Expiration Date
Sector
Employer
Union
Union Local
NAICS
Number of Workers
Committee Chair
Committee Co-Chair
Committee Member
Cochran, Moncrieff Mitchell