Small-sample comparisons of confidence intervals for the difference of two independent binomial proportions

Thomas J. Santner*, Vivek Pradhan, Pralay Senchaudhuri, Cyrus R. Mehta, Ajit Tamhane

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

23 Scopus citations


This paper compares the exact small-sample achieved coverage and expected lengths of five methods for computing the confidence interval of the difference of two independent binomial proportions. We strongly recommend that one of these be used in practice. The first method we compare is an asymptotic method based on the score statistic (AS) as proposed by Miettinen and Nurminen [1985. Comparative analysis of two rates. Statist. Med. 4, 213-226.]. Newcombe [1998. Interval estimation for the difference between independent proportions: comparison of seven methods. Statist. Med. 17, 873-890.] has shown that under a certain asymptotic set-up, confidence intervals formed from the score statistic perform better than those formed from the Wald statistic (see also [Farrington, C.P., Manning, G., 1990. Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk. Statist. Med. 9, 1447-1454.]). The remaining four methods compared are the exact methods of Agresti and Min (AM), Chan and Zhang (CZ), Coe and Tamhane (CT), and Santner and Yamagami (SY). We find that the CT has the best small-sample performance, followed by AM and CZ. Although AS is claimed to perform reasonably well, it performs the worst in this study; about 50% of the time it fails to achieve nominal coverage even with moderately large sample sizes from each binomial treatment.

Original languageEnglish (US)
Pages (from-to)5791-5799
Number of pages9
JournalComputational Statistics and Data Analysis
Issue number12
StatePublished - Aug 15 2007


  • Confidence interval
  • Exact coverage
  • P-value
  • Score statistic
  • Small-sample
  • Two-armed bandit
  • Wald statistic

ASJC Scopus subject areas

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics


Dive into the research topics of 'Small-sample comparisons of confidence intervals for the difference of two independent binomial proportions'. Together they form a unique fingerprint.

Cite this