Lessons from a partially controlled field trial

Objectives: Using the case of an on-going work-oriented prisoner-reentry experiment in Milwaukee, describe the challenges of organizing and sustaining a high-quality trial in the field in which only the randomization and data analysis are directly "controlled" by the evaluation team. Methods: The case study is of a randomized experiment involving youthful male prisoners with a history of violence and gang membership, scheduled for release into Milwaukee. The intervention included six months of pre-release services with a work-release opportunity, and intensive services and supervision following release. The case study describes the initial experimental plan and how much of that plan could be salvaged in the face of delays, administrative errors, and other problems. Results: The initial plan, when compared with the actual experiment, specified a larger and more homogeneous sample, more resources devoted to various aspects of the treatment, and more intensive supervision following release. These problems arose despite the best efforts of public officials. Randomization was preserved, and for that reason the results will still be of interest, although perhaps under-powered. Conclusions: The "gold standard" may become a bit tarnished in the field. It was crucial in this experiment to have a member of the experimental team engaged with the relevant state agencies at every step of the process to sustain this effort and to ensure that the treatment was delivered and relevant data generated. A newsletter and regular meetings with agents proved useful. The outcomes will have high internal validity.