## Abstract

This chapter explains a new approach that survey samplers can use when designing probability samples for survey experiments where there is a possibility of treatment heterogeneity. It begins by explaining why probability samples are preferred to nonprobability samples for estimating two quantities (or estimands): population average treatment effects and treatment effects within subgroups. The chapter furthermore explains why typical probability sampling methods that optimize statistical power for the average effect in a population do not necessarily optimize statistical power for the subgroup effects of interest - especially when one-s interest is in estimating effects within a rare subgroup. Next, it explains why even large, well-constructed, highly representative probability samples with randomized treatments can produce confounded analyses of differences across subgroups. The chapter illustrates the proposed approach using an empirical case study of a survey-administered behavioral science intervention: The US National Study of Learning Mindsets.

Original language | English (US) |
---|---|

Title of host publication | Experimental Methods in Survey Research |

Subtitle of host publication | Techniques that Combine Random Sampling with Random Assignment |

Publisher | wiley |

Pages | 435-456 |

Number of pages | 22 |

ISBN (Electronic) | 9781119083771 |

ISBN (Print) | 9781119083740 |

DOIs | |

State | Published - Sep 30 2019 |

## Keywords

- Empirical case study
- Population average treatment effects
- Probability samples
- Randomized treatment
- Subgroup treatment effects
- Survey experiments
- Survey-administered behavioral science intervention
- Treatment heterogeneity
- US national study of learning mindsets

## ASJC Scopus subject areas

- General Social Sciences