Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review
- PMID: 23861749
- PMCID: PMC3702538
- DOI: 10.1371/journal.pone.0066844
Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review
Abstract
Background: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.
Methodology/principal findings: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.
Conclusions: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Conflict of interest statement
Figures
Similar articles
-
Systematic review of the empirical evidence of study publication bias and outcome reporting bias.PLoS One. 2008 Aug 28;3(8):e3081. doi: 10.1371/journal.pone.0003081. PLoS One. 2008. PMID: 18769481 Free PMC article. Review.
-
Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
-
Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.Cochrane Database Syst Rev. 2014 Oct 1;2014(10):MR000035. doi: 10.1002/14651858.MR000035.pub2. Cochrane Database Syst Rev. 2014. PMID: 25271098 Free PMC article.
-
The future of Cochrane Neonatal.Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
-
Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials.Cochrane Database Syst Rev. 2014 Apr 29;2014(4):MR000034. doi: 10.1002/14651858.MR000034.pub2. Cochrane Database Syst Rev. 2014. Update in: Cochrane Database Syst Rev. 2024 Jan 4;1:MR000034. doi: 10.1002/14651858.MR000034.pub3 PMID: 24782322 Free PMC article. Updated. Review.
Cited by
-
PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews.BMJ. 2021 Mar 29;372:n160. doi: 10.1136/bmj.n160. BMJ. 2021. PMID: 33781993 Free PMC article.
-
Testing small study effects in multivariate meta-analysis.Biometrics. 2020 Dec;76(4):1240-1250. doi: 10.1111/biom.13342. Epub 2020 Aug 29. Biometrics. 2020. PMID: 32720712 Free PMC article.
-
Factors Determining Quality of Care in Family Planning Services in Africa: A Systematic Review of Mixed Evidence.PLoS One. 2016 Nov 3;11(11):e0165627. doi: 10.1371/journal.pone.0165627. eCollection 2016. PLoS One. 2016. PMID: 27812124 Free PMC article. Review.
-
Is the replication crisis a base-rate fallacy?Theor Med Bioeth. 2021 Dec;42(5-6):233-243. doi: 10.1007/s11017-022-09561-8. Epub 2022 Feb 27. Theor Med Bioeth. 2021. PMID: 35220515 Free PMC article.
-
Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review.Br J Cancer. 2018 Nov;119(10):1288-1296. doi: 10.1038/s41416-018-0305-5. Epub 2018 Oct 24. Br J Cancer. 2018. PMID: 30353050 Free PMC article.
References
-
- Song F, Parekh S, Hooper L, Loke YK, Ryder J, et al... (2010) Dissemination and publication of research findings: an updated review of related biases.. Health Technol Assess 14. - PubMed
-
- Rothstein HR, Sutton AJ, Borenstein M (2005) Publication Bias in Meta-Analysis. Prevention, Assessment and Adjustments: Wiley.
-
- Dickersin K, Min YI (1993) NIH clinical trials and publication bias. Online J Curr Clin Trials Doc No 50. - PubMed
-
- Ioannidis JP (1998) Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279: 281–286. - PubMed
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources