Please confirm you are human (Sign Up for free to never see this)
← Back to Search
Systematic Review Of The Empirical Evidence Of Study Publication Bias And Outcome Reporting Bias
K. Dwan, D. Altman, J. Arnaiz, J. Bloom, A. Chan, Eugenia Cronin, E. Decullier, P. Easterbrook, E. von Elm, C. Gamble, D. Ghersi, J. Ioannidis, J. Simes, P. Williamson
Published 2008 · Medicine
Save to my Library
Download PDFAnalyze on Scholarcy
Background The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. Methodology/Principal Findings We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. Conclusions Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
This paper references
Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses.
J. Ioannidis (2001)
Systematic reviews in health care : meta-analysis in context
M. Egger (2001)
Study of information submitted by drug companies to licensing authorities.
E. Hemminki (1980)
Quality of reporting of randomized trials as a measure of methodologic quality.
Karin Huwiler-Müntener (2002)
Evaluation of new treatments in radiation oncology: are they better than standard treatments?
H. Soares (2005)
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications
H. Melander (2003)
Identification and impact of outcome selection bias in meta-analysis.
P. Williamson (2005)
Application and investigation of a bound for outcome reporting bias
P. Williamson (2007)
Role of a research ethics committee in follow-up and publication of results
J. Pich (2003)
Reporting in randomized clinical trials improved after adoption of the CONSORT statement.
R. Kane (2007)
Bias in meta‐analysis due to outcome variable selection within studies
J. L. Hutton (2000)
Fate of biomedical research protocols and publication bias in France: retrospective cohort study
E. Decullier (2005)
Full publication of results initially presented in abstracts.
R. Scherer (2007)
Impact of funding on biomedical research: a retrospective cohort study
E. Decullier (2006)
Factors influencing the publication of health research.
Eugenia Cronin (2004)
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
A. Chan (2004)
An empirical evaluation of multifarious outcomes in pharmacogenetics: beta-2 adrenoceptor gene polymorphisms in asthma treatment
D. Contopoulos-Ioannidis (2006)
Effect of early patient enrollment on the time to completion and publication of randomized controlled trials.
A. Haidich (2001)
Issues in the design, conduct and reporting of clinical trials that impact on the quality of decision making
D. Ghersi (2006)
Publication bias in clinical trials
S. Hopewell (2007)
Publication bias in clinical research
P. Easterbrook (1991)
Publication bias in clinical trials (Protocol)
K. Loudon (2001)
Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
H. Soares (2004)
Publication bias in editorial decision making.
C. Olson (2002)
[Evaluation of clinical trials following an approval from a research ethics committee].
S. Menzel (2007)
Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments
H. Rothstein (2006)
Systematic review of publication bias in studies on publication bias
H. Dubben (2005)
Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors
A. Chan (2005)
NIH clinical trials and publication bias.
K. Dickersin (1993)
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time.
T. Trikalinos (2004)
Finding the Missing Science: The Fate of Studies Submitted for Review by a Human Subjects Committee
H. Cooper (1997)
Publication and non-publication of clinical trials: longitudinal study of applications submitted to a research ethics committee.
E. von Elm (2008)
Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
A. Chan (2004)
Systematic Reviews in Health Care
M. Egger (2001)
Publication bias and research on passive smoking: comparison of published and unpublished studies.
A. L. Misakian (1998)
Publication and related biases.
F. Song (2000)
How important is publication bias? A synthesis of available data.
K. Dickersin (1997)
Publication bias and clinical trials.
K. Dickersin (1987)
False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
I. Tannock (1996)
Outcome selection bias in meta-analysis
P. Williamson (2005)
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
J. M. Stern (1997)
Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards.
K. Dickersin (1992)
Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.
J. Ioannidis (1998)
Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee.
S. Hahn (2002)
Bias in reporting clinical trials.
A. Bardy (1998)
This paper is referenced by
Evidence-based medicine and clinical practice
M. Huic (2008)
A systematic review of the mediating role of knowledge, self-efficacy and self-care behaviour in telehealth patients with heart failure
Y. Ciere (2012)
Implications of clinical trial data sharing for medical writers
J. Ross (2013)
Individual patient data meta-analysis of time-to-event outcomes: one-stage versus two-stage approaches for estimating the hazard ratio under a random effects model.
J. Bowden (2011)
What you see depends on where you're looking and how you look at it: publication bias and outcome reporting bias.
R. Howland (2011)
Statins for multiple sclerosis.
J. Wang (2011)
Prevención secundaria: el reto permanente
V. Roger (2011)
Revisión de las tablas de composición de alimentos usadas para estimar la ingesta de nutrientes en Ecuador Revision of food composition tables used to estimate nutrient intake in Ecuador
Rocío Ortíz-Moncada (2016)
Sexual Orientation and Gender Identity/Expression Related Peer Victimization in Adolescence: A Systematic Review of Associated Psychosocial and Health Outcomes
K. Collier (2013)
The Open University ’ s repository of research publications and other research outputs Agile challenges in practice : a thematic analysis
Peggy Gregory (2015)
T . Denagamage and A . M . O ' Connor C . K . Irwin , K . J . Yoon , C . Wang ,
Chong Qing Wang (2017)
Publications for Robert Simes
R. Simes (2016)
Statistical Correction of the Winner’s Curse Explains Replication Variability in Quantitative Trait Genome-Wide Association Studies
C. Palmer (2017)
Mind-body practices: an alternative, drug-free treatment for smoking cessation? A systematic review of the literature.
Laura Carim-Todd (2013)
Conclusions erronées des résumés dans les essais cliniques contrôlés en rhumatologie : comparaisons entre les conclusions et la partie résultats des résumés
S. Mathieu (2012)
Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews
N. Pandis (2015)
Tackling submission and publication bias
Rainer Spiegel (2017)
How evidence-based medicine is failing due to biased trials and selective publication
Susanna Every-Palmer MBChB Franzcp MSc Consultant Forensic Psychiatrist (2014)
Regulators Should Better Leverage Effectiveness Standards to Enhance Drug Value
H. Naci (2014)
Pre-specification of statistical analysis approaches in published clinical trial protocols was inadequate.
L. Greenberg (2018)
Meta-assessment of bias in science
D. Fanelli (2017)
Enregistrement et publication du critère d’évaluation principal dans les essais contrôlés randomisés en oncologie
A. Boespflug (2012)
The impact of physical activity and blood pressure on cardiovascular events and mortality
A. Rossi (2015)
Nonrandomized Studies to Evaluate the Effects of a Nonpharmacological Intervention
B. Reeves (2011)
Outcomes in Clinical Trials of Inhaled Corticosteroids for Children with Asthma Are Narrowly Focussed on Short Term Disease Activity
I. Sinha (2009)
Towards improving the quality of assistive technology outcomes research.
Joshua R Tuazon (2020)
Chapter 3 – Study Design
Ü. Maiväli (2015)
A minimum core outcome dataset for the reporting of preclinical chemotherapeutic drug studies: Lessons learned from multiple discordant methodologies in the setting of colorectal cancer.
M. West (2017)
Vulnerabilities associated with wi-fi protected setup in a medical environment
D. Costantin (2017)
Intersphincteric resection: indications and outcome.
F. McDermott (2019)
Reporting quality of N-of-1 trials published between 1985 and 2013: a systematic review.
J. Li (2016)
Epidemiology of PCBs and neurodevelopment: Systematic assessment of multiplicity and completeness of reporting
M. Goodman (2020)See more