Causally valid relationships that invoke the wrong causal agent: Construct validity of the cause in policy research

Thomas D. Cook*, Yang Tang, Shari Seidman Diamond

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Internal validity relates to whether the relationship between two variables is causal. Various identification devices are used to infer the validity of this relationship, such as random assignment or regression discontinuity. However, these devices are irrelevant to validly identifying the name, label, or word that should be attached to the cause or to the effect. Yet, a valid causal relationship has little practical value for theory development or practical action unless the cause and effect are correctly labeled. This paper addresses the theory and practice of correctly labeling interventions. We examine the case study of a reform that was universally labeled as video contact with a judge during bail bond hearings. This reform replaced the practice of face-to-face contact between the judge and person charged. A daily comparative interrupted time series with 530 judges over 17 years shows that various indicators of managerial performance improved after the reform but that bail amounts unexpectedly increased. After almost 9 years, the reform was abandoned. In part, this was because the higher bail amounts were thought to result from the video method used to hear cases and not because of the case details themselves, thus compromising fair treatment under the law. A small subset of judges was chosen to implement the new video system, and analysis of the time-varying composition of judges shows that the selected judges set exceptionally high bail amounts even before the reform and also exercised a virtual monopoly of cases immediately after the reform. Since no increase in bail bond amounts was detected in longitudinal analyses of the judges hearing cases both before and after the new system was put in place, it seems that the higher bail amounts were due to time-varying changes in the kinds of judges implementing the new system and not the new system itself. If the valid increase in bail amounts had been correctly attributed to the judges rather than to the video, would the positive management gains of the new system have been foregone?.

Original languageEnglish (US)
Pages (from-to)379-414
Number of pages36
JournalJournal of the Society for Social Work and Research
Volume5
Issue number4
DOIs
StatePublished - Dec 1 2014

Keywords

  • Construct validity
  • Internal validity
  • Program evaluation

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Sociology and Political Science

Fingerprint

Dive into the research topics of 'Causally valid relationships that invoke the wrong causal agent: Construct validity of the cause in policy research'. Together they form a unique fingerprint.

Cite this