Parallel experimentation in a competitive advertising marketplace

Xiliang Lin, Harikesh S. Nair, Navdeep S. Sahni, Caio Waisman

Research output: Contribution to journalArticlepeer-review


When multiple firms are simultaneously running experiments on a platform, the treatment effects for one firm may depend on the experimentation policies of others. This paper presents a set of causal estimands that are relevant to such an environment. We also present an experimental design that is suitable for facilitating experimentation across multiple competitors in such an environment. Together, these can be used by a platform to run experiments “as a service,” on behalf of its participating firms. We show that the causal estimands we develop are identified nonparametrically by the variation induced by the design, and present two scalable estimators that help measure them in typical, high-dimensional situations. We implement the design on the advertising platform of, an eCommerce company, which is also a publisher of digital ads in China. We discuss how the design is engineered within the platform's auction-driven ad-allocation system, which is typical of modern, digital advertising marketplaces. Finally, we present results from a parallel experiment involving 16 advertisers and millions of users. These results showcase the importance of accommodating a role for interactions across experimenters and demonstrates the viability of the framework.

Original languageEnglish (US)
JournalUnknown Journal
StatePublished - Mar 26 2019
Externally publishedYes


  • A/B/n testing
  • Causal inference
  • Digital advertising
  • ECommerce
  • Experimentation
  • Platforms
  • Potential outcomes

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'Parallel experimentation in a competitive advertising marketplace'. Together they form a unique fingerprint.

Cite this