Power Analysis and Meta-Analysis are Falsely Reassuring Solutions for Concerns about Statistical Power

Colloquium | April 19 | 12:10-1:15 p.m. | 5101 Tolman Hall

 Leif Nelson, Professor, Haas School of Business

 Institute of Personality and Social Research

Transparency, disclosure, and preregistration have revealed that some statistically underpowered studies may be propped up by selective reporting and p-hacking. Conscientious researchers therefore have renewed investment in statistical power. One approach is to conduct traditional power analysis to correctly set the sample size. I argue that this tool is not helpful and often biased toward samples which are too small. A different approach is to conduct underpowered studies, but meta-analyze across a collection. I argue that this tool is strikingly prone to false-positive inferences. Statistical power is a hugely important problem, but the most widely suggested tools do not offer a helpful solution.

 ipsr@berkeley.edu, 510-642-5050