Multiple testing of general contrasts: Truncated closure and the extended shaffer-royen method

Peter H. Westfall, Randall D. Tobias

Research output: Contribution to journalArticlepeer-review

39 Scopus citations


Powerful improvements are possible for multiple testing procedures when the hypotheses are logically related. Closed testing with a-exhaustive tests provides a unifying framework for developing such procedures, but can be computationally difficult and can be "non-monotonic in p values." Royen introduced a "truncated" closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonie in p values. Shaffer developed a similar truncated procedure for more general comparisons, but using Bonferroni tests rather than α-exhaustive tests, and Westfall extended Shaffer's method to allow α-exhaustive tests. This article extends Royen's method to general contrasts and proves that it is equivalent to the extended Shaffer procedure. For k contrasts, the method generally requires evaluation of O(2k) critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k. The set of intersections is represented using a tree structure, and a branch-and-bound algorithm is used to search the tree and reduce the O(2 k) complexity by obtaining conservative "covering sets" that retain control of the familywise type I error rate. The procedure becomes less conservative as the tree search deepens, but computation time increases. In some cases where hypotheses are logically restricted, even the more conservative covering sets provide much more power than standard methods. The methods described herein are general, computable, and often much more powerful than commonly used methods for multiple testing of general contrasts, as shown by applications to pairwise comparisons and response surfaces. In particular, with response surface tests, the method is computable with complete tree search, even when k is large.

Original languageEnglish
Pages (from-to)487-494
Number of pages8
JournalJournal of the American Statistical Association
Issue number478
StatePublished - Jun 2007


  • All subsets
  • Branch-and-bound algorithm
  • Critical value
  • FWE control
  • Pairwise comparisons
  • Response surface


Dive into the research topics of 'Multiple testing of general contrasts: Truncated closure and the extended shaffer-royen method'. Together they form a unique fingerprint.

Cite this