Improving sensitivity of cluster-based permutation test for EEG/MEG data

To solve multiple comparisons problems in EEG/MEG analyses, cluster-based permutation test is possibly the most powerful approach, while it also inherits the advantage of well-controlled family-wise error rate from point-level permutation test. Because the cluster-level statistics used accumulate statistical power of all points in a cluster, cluster-based permutation test has a much higher sensitivity for widespread clusters. In this study, we demonstrate that, when the threshold for cluster inclusion is inappropriately set, the existence of larger clusters lowers the sensitivity for detecting the presence of smaller clusters, because the influence of large clusters on permutation distribution is overlooked in previous studies. Further, we demonstrated that increasing the threshold for cluster inclusion can efficiently solve this problem and then proposed a new guideline for threshold selection in the cluster-based permutation test. Results on simulated data and real data show the proposed guideline can greatly improve the sensitivity of cluster-based permutation test for detecting small clusters while retaining the same family-wise error rate.