Power, Reliability, and Heterogeneous Results
http://www.100md.com
《科学公立图书馆医学》
1 McGill University Montreal, Quebec, Canada
I want to congratulate John P. A. Ioannidis on his thought-provoking Essay [1]. I have two comments.
In Corollary 1, he suggests that small sample sizes mean smaller power, and implies that larger studies with thousands of subjects are more likely to be true. I think it is important to stress that if the effect size is large (e.g., very small variance that is seen in physiological studies), then adequate power is obtained with small numbers. Furthermore, some would argue that exposing subjects to research risks unnecessarily (e.g., when fewer subjects would yield sufficient power) is unethical. Since the analysis is based on power, we should remember that larger is not always better.
In Corollary 4, Ioannidis argues that greater flexibility in designs, definitions, etc. means the results are less likely to be true. I agree that replication of all aspects of the study is more likely to yield consistent results, but this does not necessarily mean true results. Since we don't know a priori which methodological details are most appropriate (e.g., dose, timing, etc.), heterogeneous results from different designs is an important source of information and can lead to a new, more in-depth understanding of the subject—and sometimes even paradigm shifts. I agree with the accompanying Editorial [2] to the article that we need to distinguish between the validity of the data and the validity of the authors' conclusions.
References
Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2:e124 DOI: 10.1371/journal.pmed.0020124.
PLoS Medicine Editors. (2005) Minimizing mistakes and embracing uncertainty. PLoS Med 2:e272 DOI: 10.1371/journal.pmed.0020272.(Ian Shrier)
I want to congratulate John P. A. Ioannidis on his thought-provoking Essay [1]. I have two comments.
In Corollary 1, he suggests that small sample sizes mean smaller power, and implies that larger studies with thousands of subjects are more likely to be true. I think it is important to stress that if the effect size is large (e.g., very small variance that is seen in physiological studies), then adequate power is obtained with small numbers. Furthermore, some would argue that exposing subjects to research risks unnecessarily (e.g., when fewer subjects would yield sufficient power) is unethical. Since the analysis is based on power, we should remember that larger is not always better.
In Corollary 4, Ioannidis argues that greater flexibility in designs, definitions, etc. means the results are less likely to be true. I agree that replication of all aspects of the study is more likely to yield consistent results, but this does not necessarily mean true results. Since we don't know a priori which methodological details are most appropriate (e.g., dose, timing, etc.), heterogeneous results from different designs is an important source of information and can lead to a new, more in-depth understanding of the subject—and sometimes even paradigm shifts. I agree with the accompanying Editorial [2] to the article that we need to distinguish between the validity of the data and the validity of the authors' conclusions.
References
Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2:e124 DOI: 10.1371/journal.pmed.0020124.
PLoS Medicine Editors. (2005) Minimizing mistakes and embracing uncertainty. PLoS Med 2:e272 DOI: 10.1371/journal.pmed.0020272.(Ian Shrier)