Online citations, reference lists, and bibliographies.

Empirical Evidence For Selective Reporting Of Outcomes In Randomized Trials: Comparison Of Protocols To Published Articles.

A. Chan, A. Hrõbjartsson, M. T. Haahr, P. Gøtzsche, D. Altman
Published 2004 · Medicine
Cite This
Download PDF
Analyze on Scholarcy
Share
CONTEXT Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports. OBJECTIVE To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials. DESIGN Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes. MAIN OUTCOME MEASURES Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles. RESULTS One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary. CONCLUSIONS The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion.
This paper references
10.4135/9781848608344.n21
Publication and related biases.
F. Song (2000)
The power of the proto
JP Siegel (2002)
10.1353/pla.2003.0039
Institutional Repositories: Essential Infrastructure For Scholarship In The Digital Age
C. Lynch (2003)
10.1093/jnci/88.3-4.206
False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
I. Tannock (1996)
Commentary: Changes Between Protocol And Manuscript Should Be Declared At Submission
Sandra Goldbeck-Wood (2001)
10.1136/bmj.280.6217.833
Study of information submitted by drug companies to licensing authorities.
E. Hemminki (1980)
Commentary: Research Governance Must Focus On Research Training
G. Murray (2001)
How important is publication bias? A synthesis of available data.
K. Dickersin (1997)
10.1046/j.1365-2753.2000.00236.x
Individual patient data meta-analysis of randomized anti-epileptic drug monotherapy trials.
P. Williamson (2000)
10.1002/1097-0258(20001230)19:24<3325::AID-SIM827>3.0.CO;2-D
Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies.
S. Hahn (2000)
10.1007/s00784-002-0188-x
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
D. Moher (2003)
10.1186/2048-4623-1-s3-pa005
Outcome reporting bias and individual patient data meta-analysis: a case study in surgery
Kirsty McCormack (2001)
10.1002/14651858.CD001319.PUB5
Colloid solutions for fluid resuscitation.
F. Bunn (2001)
Colloid solutions for fluid resuscitation [Cochrane Review on CDROM
F Bunn (2002)
Agreement on cooperation on clinical trials between physicians and the pharmaceutical industry
J P Siegel
10.1016/0197-2456(89)90017-2
Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.
Peter Christian Gøtzsche (1989)
10.1186/1471-8219-2-4
Publishing study protocols: making them visible will improve registration, reporting and recruitment
Fiona Godlee (2001)
10.1056/nejm199011083231920
Editorial review of protocols for clinical trials.
Jay P. Siegel (1990)
Publication bias in statistical overview of trials: example of psychological rehabilitation following myocardial infarction [abstract
RR West (1997)
10.1002/art.1780330119
Time for changes in the design, analysis, and reporting of rheumatoid arthritis clinical trials.
David T Felson (1990)
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical in SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS 2464 JAMA, May 26, 2004—Vol
H Melander (2004)
10.1136/bmj.323.7324.1309
Journals should see original protocols for clinical trials
C. Hawkey (2001)
Goldbeck-Wood S. Changes between protocol and manuscript should be declared at submission
C A Silagy (2001)
10.1001/jama.287.21.2831
Publishing protocols of systematic reviews: comparing what was done to what was planned.
Chris A Silagy (2002)
10.7326/0003-4819-134-8-200104170-00012
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
D. Altman (2001)
Outcome reporting bias and individual patient data metaanalysis: a case study in surgery [abstract]. In: Abstracts for Workshops and Scientific Sessions
K McCormack (2001)
10.1016/S0140-6736(98)07618-1
How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing
I. Chalmers (1999)
[Data torturing].
A Kohn (1994)
10.1136/bmj.316.7126.221
Meta-analysis: Unresolved issues and future developments
G. Smith (1998)
10.1016/0895-4356(92)90072-U
Bias in meta-analytic research.
D. Felson (1992)
10.1258/1355819021927674
Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice
J. Higgins (2002)
10.1016/S0140-6736(02)11652-7
The power of the protocol
Marissa Nichole Lassere (2002)
10.1046/j.1365-2753.2002.00314.x
Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee.
S. Hahn (2002)
10.1056/NEJM198708133170706
Statistical problems in the reporting of clinical trials. A survey of three medical journals.
S. Pocock (1987)
10.1136/bmj.328.7430.22
Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
H. Soares (2004)
Meta-analysis of controlled clinical trials.
C. David Naylor (1989)
Evidence b ( i ) ased medicine — selective reporting from studies sponsored by pharmaceutical in - SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS
H Melander
10.1136/bmj.326.7400.1171
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications
H. Melander (2003)
10.1001/jama.1990.03440100121018
Underreporting research is scientific misconduct.
I. Chalmers (1990)



This paper is referenced by
Syddansk Universitet Promoting public access to clinical trial protocols Challenges and recommendations
An-Wen Chan (2018)
10.1038/oby.2008.333
Review and meta-analysis of pharmacotherapy for binge-eating disorder.
D. L. Reas (2008)
Exercise and BoneMineral Density in PremenopausalWomen: AMeta-Analysis of Randomized Controlled Trials
George A. Kelley (2015)
10.1016/j.jclinepi.2018.05.023
Pre-specification of statistical analysis approaches in published clinical trial protocols was inadequate.
Lauren Greenberg (2018)
10.1016/S2215-0366(18)30269-4
Comparative efficacy and tolerability of medications for attention-deficit hyperactivity disorder in children, adolescents, and adults: a systematic review and network meta-analysis
Samuele Cortese (2018)
10.23736/S1973-9087.19.05807-6
The human risks of bias in medical and rehabilitation research and practice: the eight Is.
Antti Malmivaara (2019)
10.3233/JRS-200006
What have antidepressants been tested for? A systematic review.
Peter Christian Gøtzsche (2020)
10.1002/sim.7815
Multivariate network meta-analysis to mitigate the effects of outcome reporting bias.
Hyunsoo Hwang (2018)
10.1016/S1134-2072(09)70485-2
Resúmenes de ensayos clínicos: recomendaciones para su elaboración
José Jiménez Villa (2009)
10.1093/ije/dyn265
Commentary: Which meta-analyses are conclusive?
Eveline Nüesch (2009)
10.1093/ije/dyn188
Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses.
Jesper Brok (2009)
10.1007/s00167-010-1365-0
Patellar resurfacing in total knee arthroplasty for osteoarthritis: a meta-analysis
Yonghui Fu (2010)
10.1002/9781118156391.EBCP002002
Developing Clinical Guidelines for Adults
Stephen Pilling (2012)
10.2139/SSRN.2718888
Managing Intellectual Property Rights Over Clinical Trial Data to Promote Access and Benefit Sharing in Public Health
P. Andanda (2013)
10.1371/journal.pmed.1001419
PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts
Elaine M. Beller (2013)
10.1136/bmj.c869
CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials
David Moher (2010)
10.1136/jme.2010.035378
Discrepancies between research advertisements and disclosure of study locations in trial registrations for USA-sponsored research in Russia
Daniel Patrone (2010)
10.1097/EJA.0000000000000084
Why should clinical trials be registered?
Elizabeth Wager (2014)
10.1186/S12930-014-0010-3
Subjects with colour vision deficiency in the community: what do primary care physicians need to know?
Xin Bei V Chan (2014)
10.1016/j.jclinepi.2014.05.007
The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications.
Segun Raphael Bello (2014)
10.1371/journal.pone.0109400
Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability
I. Saldanha (2014)
10.2217/cer.14.31
Should comparative effectiveness research ignore industry-funded data?
Adam G. Dunn (2014)
10.1007/978-1-4939-3477-5_10
Evidence Mapping to Advance Justice Practice
Michael S. Caudy (2016)
10.1016/j.annepidem.2016.07.001
Outcome reporting bias in observational epidemiology studies on phthalates.
Gerard M. H. Swaen (2016)
10.1016/j.jclinepi.2016.05.016
Randomized trials are frequently fragmented in multiple secondary publications.
Shanil Ebrahim (2016)
10.1086/698030
Experimental Design for Laboratory Biologists: Maximising Information and Improving Reproducibility
Stanley E. Lazic (2016)
10.1001/jama.2013.879
Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension.
Melanie J Calvert (2013)
10.1007/s10151-017-1585-0
Laparoscopic lavage versus surgical resection for acute diverticulitis with generalised peritonitis: a systematic review and meta-analysis
Roberto Cirocchi (2017)
10.1371/journal.pone.0167709
Non-Publication Is Common among Phase 1, Single-Center, Not Prospectively Registered, or Early Terminated Clinical Drug Trials
Cornelis A van den Bogert (2016)
10.1016/j.jclinepi.2016.08.002
Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials.
Segun Raphael Bello (2017)
10.1038/nbt.3809
Evidence of insufficient quality of reporting in patent landscapes in the life sciences
James A Smith (2017)
10.1503/cjs.010616
Are outcomes reported in surgical randomized trials patient-important? A systematic review and meta-analysis
S. Adie (2017)
See more
Semantic Scholar Logo Some data provided by SemanticScholar