TY - JOUR
T1 - Multiple testing of general contrasts
T2 - Truncated closure and the extended shaffer-royen method
AU - Westfall, Peter H.
AU - Tobias, Randall D.
PY - 2007/6
Y1 - 2007/6
N2 - Powerful improvements are possible for multiple testing procedures when the hypotheses are logically related. Closed testing with a-exhaustive tests provides a unifying framework for developing such procedures, but can be computationally difficult and can be "non-monotonic in p values." Royen introduced a "truncated" closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonie in p values. Shaffer developed a similar truncated procedure for more general comparisons, but using Bonferroni tests rather than α-exhaustive tests, and Westfall extended Shaffer's method to allow α-exhaustive tests. This article extends Royen's method to general contrasts and proves that it is equivalent to the extended Shaffer procedure. For k contrasts, the method generally requires evaluation of O(2k) critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k. The set of intersections is represented using a tree structure, and a branch-and-bound algorithm is used to search the tree and reduce the O(2 k) complexity by obtaining conservative "covering sets" that retain control of the familywise type I error rate. The procedure becomes less conservative as the tree search deepens, but computation time increases. In some cases where hypotheses are logically restricted, even the more conservative covering sets provide much more power than standard methods. The methods described herein are general, computable, and often much more powerful than commonly used methods for multiple testing of general contrasts, as shown by applications to pairwise comparisons and response surfaces. In particular, with response surface tests, the method is computable with complete tree search, even when k is large.
AB - Powerful improvements are possible for multiple testing procedures when the hypotheses are logically related. Closed testing with a-exhaustive tests provides a unifying framework for developing such procedures, but can be computationally difficult and can be "non-monotonic in p values." Royen introduced a "truncated" closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonie in p values. Shaffer developed a similar truncated procedure for more general comparisons, but using Bonferroni tests rather than α-exhaustive tests, and Westfall extended Shaffer's method to allow α-exhaustive tests. This article extends Royen's method to general contrasts and proves that it is equivalent to the extended Shaffer procedure. For k contrasts, the method generally requires evaluation of O(2k) critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k. The set of intersections is represented using a tree structure, and a branch-and-bound algorithm is used to search the tree and reduce the O(2 k) complexity by obtaining conservative "covering sets" that retain control of the familywise type I error rate. The procedure becomes less conservative as the tree search deepens, but computation time increases. In some cases where hypotheses are logically restricted, even the more conservative covering sets provide much more power than standard methods. The methods described herein are general, computable, and often much more powerful than commonly used methods for multiple testing of general contrasts, as shown by applications to pairwise comparisons and response surfaces. In particular, with response surface tests, the method is computable with complete tree search, even when k is large.
KW - All subsets
KW - Branch-and-bound algorithm
KW - Critical value
KW - FWE control
KW - Pairwise comparisons
KW - Response surface
UR - http://www.scopus.com/inward/record.url?scp=34250724822&partnerID=8YFLogxK
U2 - 10.1198/016214506000001338
DO - 10.1198/016214506000001338
M3 - Article
AN - SCOPUS:34250724822
SN - 0162-1459
VL - 102
SP - 487
EP - 494
JO - Journal of the American Statistical Association
JF - Journal of the American Statistical Association
IS - 478
ER -