Manipulating the alpha level cannot cure significance testing

• DOI: 10.7287/peerj.preprints.3411v2
• Publication Date: 2018-04-11T15:25:37Z
• Authors: David Trafimow; Valentin Amrhein; Corson N. Areshen...; Carlos Barrera-Ca...; Eric J. Beh; Yusuf Bilgiç; Roser Bono; Michael T. Bradley; William M. Briggs; Héctor A. Cepeda-...; Sergio E. Chaigneau; Daniel R. Ciocca; Juan Carlos Correa; Denis Cousineau; Michiel R. de Boer; Subhra Sankar Dhar; Igor Dolgov; Juana Gómez-Benito; Marian Grendar; James Grice; Martin E. Guerrer...; Andrés Gutiérrez; Tania B. Huedo-Me...; Klaus Jaffe; Armina Janyan; Ali Karimnezhad; Fränzi Korner-Nie...; Koji Kosugi; Martin Lachmair; Rubén Ledesma; Roberto Limongi; Marco Tullio Liuzza; Rosaria Lombardo; Michael Marks; Gunther Meinlschmidt; Ladislas Nalborczyk; Hung T. Nguyen; Raydonal Ospina; Jose D. Perezgonz...; Roland Pfister; Juan José Rahona; David A. Rodrígue...; Xavier Romão; Susana Ruiz-Ferná...; Isabel Suarez; Marion Tegethoff; Mauricio Tejo; Rens van de Schoot; Ivan Vankov; Santiago Velasco-...; Tonghui Wang; Yuki Yamada; Felipe C. Zoppino; Fernando Marmolej...
• Keywords: Significance testing; Statistical Inference; Multiple comparisons problem; Value (mathematics)

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = .05 to .005, is deleterious finding of new discoveries and progress science. Given blanket variable levels both are problematic, it sensible dispense with significance testing altogether. There alternatives address study design sample size much more directly than does; but none statistical tools should be taken as magic method giving clear-cut mechanical answers. Inference not based single studies at all, cumulative evidence multiple independent studies. When evaluating strength evidence, we consider, example, auxiliary assumptions, experimental design, implications applications. To boil all this down binary decision -value threshold .05, .01, or anything else, acceptable.