Seeing the Forest Despite the Trees: The Benefit of Exploratory Data Analysis to Program Evaluation Research

In the present article, it is argued that there is a benefit to applying techniques of exploratory data analysis (EDA) to program evaluation. To exemplify this, an evaluation of a rehabilitation program for people with rheumatoid arthritis ispresented. The perceived health status of patients receiving intensive rehabilitation services from a major rehabilitation institute was compared with that of patients receiving customary office-based care over an 18-month period. The data were analyzed in a conventional way (analysis of variance) and then by way of EDA techniques (graphic display of medians and boxplots). The conventional analysis suggested that all patients improved over time and that intensive rehabilitation services provided no particular benefit or harm. The exploratory analysis showed that the distribution of the outcome variable was patently nonnormal, thus casting doubt on the validity of the conventional analysis. The EDA furthershowed that the rehabilitation group lagged behind the comparison group fora year, with a precipitious improvementat the 18-month period. Thissuggests that a selection factor was operating (i.e., those in the rehabilitation group could have been sicker) or that the patients in the rehabilitation group were made more aware of their condition by the intensive health services they received The EDA provided an important insight.