A step-up multiple test procedure

C. W. Dunnett, A. Tamhane

Research output: Contribution to journalArticlepeer-review

184 Scopus citations

Abstract

We consider the problem of simultaneously testing k ≥ 2 hypotheses on parameters θ1, ..., θk. In a typical application the θ's may be a set of contrasts, for instance, a set of orthogonal contrasts among population means or a set of differences between k treatment means and a standard treatment mean. It is assumed that least squares estimators are available that are jointly normally distributed with a common variance (known up to a scalar, namely the error variance σ2) and a common known correlation. An independent χ2-distributed unbiased estimator of σ2 is also assumed to be available. We propose a step-up multiple test procedure for this problem which tests the t statistics for the k hypotheses in order starting with the least significant one and continues as long as an acceptance occurs. (By contrast, the step-down approach, which is usually used, starts with the most significant and continues as long as a rejection occurs.) Critical constants required by this step-up procedure to control the type I familywise error rate at or below a specified level α are computed for both one-sided and two-sided testing problems. Power comparisons are made for one-sided testing problems with the well-known normal theory based single-step and step-down test procedures, and also with a step-up test procedure proposed for a wider class of problems by Hochberg. (Two improvements over Hochberg's procedure by Hommel and Rom provide at best marginal increases in power, with the former being also more difficult to apply, and hence they are not included here.) Two different definitions of power are considered--the probability of rejecting all false hypotheses and the probability of rejecting at least one false hypothesis; the results are found to be qualitatively similar. The proposed step-up procedure is more powerful than the single-step procedure except when only one hypothesis is false, in which case it is slightly less powerful. Similarly, it is slightly less powerful than the step-down procedure when a few hypotheses are false, but it is more powerful when most or all hypotheses are false, and this power advantage increases with the number of such hypotheses under test. The proposed step-up procedure is uniformly more powerful than the Hochberg procedure and its improvements. A disadvantage of the proposed step-up procedure is the greater difficulty of computing its critical points. These are given for one-sided and two-sided tests for 5% level of significance.
Original languageEnglish
Pages (from-to)162-170
JournalJournal of the American Statistical Association
Volume87
DOIs
StatePublished - 1992

Fingerprint Dive into the research topics of 'A step-up multiple test procedure'. Together they form a unique fingerprint.

Cite this