TY - JOUR

T1 - Multiple testing of general contrasts

T2 - Truncated closure and the extended shaffer-royen method

AU - Westfall, Peter H.

AU - Tobias, Randall D.

PY - 2007/6

Y1 - 2007/6

N2 - Powerful improvements are possible for multiple testing procedures when the hypotheses are logically related. Closed testing with a-exhaustive tests provides a unifying framework for developing such procedures, but can be computationally difficult and can be "non-monotonic in p values." Royen introduced a "truncated" closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonie in p values. Shaffer developed a similar truncated procedure for more general comparisons, but using Bonferroni tests rather than α-exhaustive tests, and Westfall extended Shaffer's method to allow α-exhaustive tests. This article extends Royen's method to general contrasts and proves that it is equivalent to the extended Shaffer procedure. For k contrasts, the method generally requires evaluation of O(2k) critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k. The set of intersections is represented using a tree structure, and a branch-and-bound algorithm is used to search the tree and reduce the O(2 k) complexity by obtaining conservative "covering sets" that retain control of the familywise type I error rate. The procedure becomes less conservative as the tree search deepens, but computation time increases. In some cases where hypotheses are logically restricted, even the more conservative covering sets provide much more power than standard methods. The methods described herein are general, computable, and often much more powerful than commonly used methods for multiple testing of general contrasts, as shown by applications to pairwise comparisons and response surfaces. In particular, with response surface tests, the method is computable with complete tree search, even when k is large.

AB - Powerful improvements are possible for multiple testing procedures when the hypotheses are logically related. Closed testing with a-exhaustive tests provides a unifying framework for developing such procedures, but can be computationally difficult and can be "non-monotonic in p values." Royen introduced a "truncated" closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonie in p values. Shaffer developed a similar truncated procedure for more general comparisons, but using Bonferroni tests rather than α-exhaustive tests, and Westfall extended Shaffer's method to allow α-exhaustive tests. This article extends Royen's method to general contrasts and proves that it is equivalent to the extended Shaffer procedure. For k contrasts, the method generally requires evaluation of O(2k) critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k. The set of intersections is represented using a tree structure, and a branch-and-bound algorithm is used to search the tree and reduce the O(2 k) complexity by obtaining conservative "covering sets" that retain control of the familywise type I error rate. The procedure becomes less conservative as the tree search deepens, but computation time increases. In some cases where hypotheses are logically restricted, even the more conservative covering sets provide much more power than standard methods. The methods described herein are general, computable, and often much more powerful than commonly used methods for multiple testing of general contrasts, as shown by applications to pairwise comparisons and response surfaces. In particular, with response surface tests, the method is computable with complete tree search, even when k is large.

KW - All subsets

KW - Branch-and-bound algorithm

KW - Critical value

KW - FWE control

KW - Pairwise comparisons

KW - Response surface

UR - http://www.scopus.com/inward/record.url?scp=34250724822&partnerID=8YFLogxK

U2 - 10.1198/016214506000001338

DO - 10.1198/016214506000001338

M3 - Article

AN - SCOPUS:34250724822

SN - 0162-1459

VL - 102

SP - 487

EP - 494

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

IS - 478

ER -