Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences

• DOI: 10.31219/osf.io/w3s3s
• Publication Date: 2018-07-02T10:51:37Z
• Authors: Felix D. Schönbrodt; Eric-Jan Wagenmakers; Michael Zehetleitner; Marco Perugini
• Keywords: Sequential analysis; Intuition; Alternative hypothesis; Multiple comparisons problem

Unplanned optional stopping rules have been criticized for inflating Type I error rates under the null hypothesis significance testing (NHST) paradigm. Despite these criticisms this research practice is not uncommon, probably as it appeals to researcher’s intuition collect more data in order push an indecisive result into a decisive region. In contribution we investigate properties of procedure Bayesian that allows with unlimited multiple testing, even after each participant. procedure, which call Sequential Bayes Factors (SBF), factors are computed until priori defined level evidence reached. This flexible sampling plans and dependent upon correct effect size guesses power analysis. We investigated long-term rate misleading evidence, average expected sample sizes, biasedness estimates when SBF design applied test mean differences between two groups. Compared optimal NHST, typically needs 50% 70% smaller samples reach conclusion about presence effect, while having same or lower wrong inference.