The trouble with instruments: The need for pretreatment balance in shock-based instrumental variable designs

Credible causal inference in accounting and finance research often comes from natural experiments. These experiments can be exploited using several shock-based research designs, including difference in differences (DID), shock-based instrumental variable (shock-IV), and regression discontinuity. We study here shock-IV designs using panel data. We identify all shock-IV papers in two broad data sets and reexamine three of the apparently strongest papers—Desai and Dharmapala [Desai M, Dharmapala D (2009) Corporate tax avoidance and firm value. Rev. Econom. Statist. 91:537–546.], Duchin et al. [Duchin R, Matsusaka J, Ozbas O (2010) When are outside directors effective? J. Financial Econom. 95:195–214.], and Iliev [Iliev P (2010) The effect of SOX Section 404: Costs, earnings quality, and stock prices. J. Finance 65:1163–1196.]. After we enforce covariate balance and common support for treated and control firms, the instruments in all three papers are unusable—they are no longer significant in the first stage. All three papers also show nonparallel pretreatment trends on outcomes or core covariates. The problems with these papers generalize to our full sample and to other papers exploiting the same shocks as Duchin et al. A core conclusion of our reexamination is that pretreatment balance (common support, covariate balance, and parallel pretreatment trends) is necessary for credible shock-IV designs. We provide a good-practice checklist for shock-IV design with panel data, much of which also applies to DID designs.