Online citations, reference lists, and bibliographies.

Empirical Evidence For Selective Reporting Of Outcomes In Randomized Trials: Comparison Of Protocols To Published Articles.

A. Chan, A. Hrõbjartsson, M. T. Haahr, P. Gøtzsche, D. Altman
Published 2004 · Medicine
Cite This
Download PDF
Analyze on Scholarcy
CONTEXT Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports. OBJECTIVE To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials. DESIGN Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes. MAIN OUTCOME MEASURES Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles. RESULTS One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary. CONCLUSIONS The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion.
This paper references
Publication and related biases.
F. Song (2000)
The power of the proto
JP Siegel (2002)
Institutional Repositories: Essential Infrastructure For Scholarship In The Digital Age
C. Lynch (2003)
False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
I. Tannock (1996)
Commentary: Changes Between Protocol And Manuscript Should Be Declared At Submission
Sandra Goldbeck-Wood (2001)
Study of information submitted by drug companies to licensing authorities.
E. Hemminki (1980)
Commentary: Research Governance Must Focus On Research Training
G. Murray (2001)
How important is publication bias? A synthesis of available data.
K. Dickersin (1997)
Individual patient data meta-analysis of randomized anti-epileptic drug monotherapy trials.
P. Williamson (2000)
Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies.
S. Hahn (2000)
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
D. Moher (2003)
Outcome reporting bias and individual patient data meta-analysis: a case study in surgery
Kirsty McCormack (2001)
Colloid solutions for fluid resuscitation.
F. Bunn (2001)
Colloid solutions for fluid resuscitation [Cochrane Review on CDROM
F Bunn (2002)
Agreement on cooperation on clinical trials between physicians and the pharmaceutical industry
J P Siegel
Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.
Peter Christian Gøtzsche (1989)
Publishing study protocols: making them visible will improve registration, reporting and recruitment
Fiona Godlee (2001)
Editorial review of protocols for clinical trials.
Jay P. Siegel (1990)
Publication bias in statistical overview of trials: example of psychological rehabilitation following myocardial infarction [abstract
RR West (1997)
Time for changes in the design, analysis, and reporting of rheumatoid arthritis clinical trials.
David T Felson (1990)
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical in SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS 2464 JAMA, May 26, 2004—Vol
H Melander (2004)
Journals should see original protocols for clinical trials
C. Hawkey (2001)
Goldbeck-Wood S. Changes between protocol and manuscript should be declared at submission
C A Silagy (2001)
Publishing protocols of systematic reviews: comparing what was done to what was planned.
Chris A Silagy (2002)
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
D. Altman (2001)
Outcome reporting bias and individual patient data metaanalysis: a case study in surgery [abstract]. In: Abstracts for Workshops and Scientific Sessions
K McCormack (2001)
How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing
I. Chalmers (1999)
[Data torturing].
A Kohn (1994)
Meta-analysis: Unresolved issues and future developments
G. Smith (1998)
Bias in meta-analytic research.
D. Felson (1992)
Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice
J. Higgins (2002)
The power of the protocol
Marissa Nichole Lassere (2002)
Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee.
S. Hahn (2002)
Statistical problems in the reporting of clinical trials. A survey of three medical journals.
S. Pocock (1987)
Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
H. Soares (2004)
Meta-analysis of controlled clinical trials.
C. David Naylor (1989)
Evidence b ( i ) ased medicine — selective reporting from studies sponsored by pharmaceutical in - SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS
H Melander
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications
H. Melander (2003)
Underreporting research is scientific misconduct.
I. Chalmers (1990)

This paper is referenced by
Syddansk Universitet Promoting public access to clinical trial protocols Challenges and recommendations
An-Wen Chan (2018)
Review and meta-analysis of pharmacotherapy for binge-eating disorder.
D. L. Reas (2008)
Exercise and BoneMineral Density in PremenopausalWomen: AMeta-Analysis of Randomized Controlled Trials
George A. Kelley (2015)
Pre-specification of statistical analysis approaches in published clinical trial protocols was inadequate.
Lauren Greenberg (2018)
Comparative efficacy and tolerability of medications for attention-deficit hyperactivity disorder in children, adolescents, and adults: a systematic review and network meta-analysis
Samuele Cortese (2018)
The human risks of bias in medical and rehabilitation research and practice: the eight Is.
Antti Malmivaara (2019)
What have antidepressants been tested for? A systematic review.
Peter Christian Gøtzsche (2020)
Multivariate network meta-analysis to mitigate the effects of outcome reporting bias.
Hyunsoo Hwang (2018)
Resúmenes de ensayos clínicos: recomendaciones para su elaboración
José Jiménez Villa (2009)
Commentary: Which meta-analyses are conclusive?
Eveline Nüesch (2009)
Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses.
Jesper Brok (2009)
Patellar resurfacing in total knee arthroplasty for osteoarthritis: a meta-analysis
Yonghui Fu (2010)
Developing Clinical Guidelines for Adults
Stephen Pilling (2012)
Managing Intellectual Property Rights Over Clinical Trial Data to Promote Access and Benefit Sharing in Public Health
P. Andanda (2013)
PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts
Elaine M. Beller (2013)
CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials
David Moher (2010)
Discrepancies between research advertisements and disclosure of study locations in trial registrations for USA-sponsored research in Russia
Daniel Patrone (2010)
Why should clinical trials be registered?
Elizabeth Wager (2014)
Subjects with colour vision deficiency in the community: what do primary care physicians need to know?
Xin Bei V Chan (2014)
The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications.
Segun Raphael Bello (2014)
Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability
I. Saldanha (2014)
Should comparative effectiveness research ignore industry-funded data?
Adam G. Dunn (2014)
Evidence Mapping to Advance Justice Practice
Michael S. Caudy (2016)
Outcome reporting bias in observational epidemiology studies on phthalates.
Gerard M. H. Swaen (2016)
Randomized trials are frequently fragmented in multiple secondary publications.
Shanil Ebrahim (2016)
Experimental Design for Laboratory Biologists: Maximising Information and Improving Reproducibility
Stanley E. Lazic (2016)
Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension.
Melanie J Calvert (2013)
Laparoscopic lavage versus surgical resection for acute diverticulitis with generalised peritonitis: a systematic review and meta-analysis
Roberto Cirocchi (2017)
Non-Publication Is Common among Phase 1, Single-Center, Not Prospectively Registered, or Early Terminated Clinical Drug Trials
Cornelis A van den Bogert (2016)
Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials.
Segun Raphael Bello (2017)
Evidence of insufficient quality of reporting in patent landscapes in the life sciences
James A Smith (2017)
Are outcomes reported in surgical randomized trials patient-important? A systematic review and meta-analysis
S. Adie (2017)
See more
Semantic Scholar Logo Some data provided by SemanticScholar