TY - GEN
T1 - The Role of Randomized Control Trials in Online Demand Generation
T2 - 42nd International Conference on Information Systems: Building Sustainability and Resilience with IS: A Call for Action, ICIS 2021 TREOs
AU - Runge, Julian
AU - Nair, Harikesh
N1 - Publisher Copyright:
© 2021 42nd International Conference on Information Systems, ICIS 2021 TREOs: "Building Sustainability and Resilience with IS: A Call for Action". All Rights Reserved.
PY - 2021
Y1 - 2021
N2 - Observing the organically evolving Facebook advertising marketplace, this paper sets out to study how the adoption of randomized control trials (RCTs) for policy evaluation impacts firms’ behavior and performance on the platform. Findings indicate strong positive effects from the adoption of RCTs: After or in coincidence with RCT adoption, firms increase their investment in demand generation, at stable or higher levels of performance, as measured by customer purchases achieved per advertising dollar spent. A human expert provided to firms by the platform is also able to induce increases in investment but smaller ones, and, importantly, performance gains appear to exclusively materialize after the adoption of RCTs for policy evaluation, and not through human expertise alone. By means of the unbiased causal estimates obtained through RCTs, firms may be able to uniquely reduce their uncertainty about the environment and identify effective policies to exploit it with certainty.
AB - Observing the organically evolving Facebook advertising marketplace, this paper sets out to study how the adoption of randomized control trials (RCTs) for policy evaluation impacts firms’ behavior and performance on the platform. Findings indicate strong positive effects from the adoption of RCTs: After or in coincidence with RCT adoption, firms increase their investment in demand generation, at stable or higher levels of performance, as measured by customer purchases achieved per advertising dollar spent. A human expert provided to firms by the platform is also able to induce increases in investment but smaller ones, and, importantly, performance gains appear to exclusively materialize after the adoption of RCTs for policy evaluation, and not through human expertise alone. By means of the unbiased causal estimates obtained through RCTs, firms may be able to uniquely reduce their uncertainty about the environment and identify effective policies to exploit it with certainty.
KW - Experimentation
KW - exploration-exploitation
KW - firm learning
KW - online advertising
KW - randomized control trials
UR - https://www.scopus.com/pages/publications/85144239705
UR - https://www.scopus.com/inward/citedby.url?scp=85144239705&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85144239705
T3 - 42nd International Conference on Information Systems, ICIS 2021 TREOs: "Building Sustainability and Resilience with IS: A Call for Action"
BT - 42nd International Conference on Information Systems, ICIS 2021 TREOs
PB - Association for Information Systems
Y2 - 12 December 2021 through 15 December 2021
ER -
