Online citations, reference lists, and bibliographies.
Please confirm you are human
(Sign Up for free to never see this)
← Back to Search

Empirical Evidence For Selective Reporting Of Outcomes In Randomized Trials: Comparison Of Protocols To Published Articles.

A. Chan, A. Hrõbjartsson, M. Haahr, P. Gøtzsche, D. Altman
Published 2004 · Medicine

Save to my Library
Download PDF
Analyze on Scholarcy
Share
CONTEXT Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports. OBJECTIVE To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials. DESIGN Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes. MAIN OUTCOME MEASURES Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles. RESULTS One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary. CONCLUSIONS The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion.
This paper references
10.1002/ART.1780330119
Time for changes in the design, analysis, and reporting of rheumatoid arthritis clinical trials.
D. Felson (1990)
10.1136/bmj.280.6217.833
Study of information submitted by drug companies to licensing authorities.
E. Hemminki (1980)
10.4135/9781848608344.n21
Publication and related biases.
F. Song (2000)
10.1056/nejm199011083231920
Editorial review of protocols for clinical trials.
J. Siegel (1990)
10.1136/BMJ.323.7324.1309
Journals should see original protocols for clinical trials
C. Hawkey (2001)
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical in SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS 2464 JAMA, May 26, 2004—Vol
H Melander (2004)
10.1186/1471-2288-1-2
The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials
D. Moher (2001)
Colloid solutions for fluid resuscitation [Cochrane Review on CDROM
F Bunn (2002)
The power of the proto
JP Siegel (2002)
Goldbeck-Wood S. Changes between protocol and manuscript should be declared at submission
C A Silagy (2001)
10.1093/JNCI/88.3-4.206
False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
I. Tannock (1996)
Outcome reporting bias and individual patient data metaanalysis: a case study in surgery [abstract]. In: Abstracts for Workshops and Scientific Sessions
K McCormack (2001)
10.1353/pla.2003.0039
Institutional Repositories: Essential Infrastructure For Scholarship In The Digital Age
C. Lynch (2003)
10.1016/S0140-6736(02)11652-7
The power of the protocol
M. Lassere (2002)
Evidence b ( i ) ased medicine — selective reporting from studies sponsored by pharmaceutical in - SELECTIVE REPORTING OF OUTCOMES IN RANDOMIZED TRIALS
H Melander
10.1258/1355819021927674
Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice
J. Higgins (2002)
10.1002/1097-0258(20001230)19:24<3325::AID-SIM827>3.0.CO;2-D
Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies.
S. Hahn (2000)
10.1007/s00784-002-0188-x
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
D. Moher (2003)
Systematic reviews in health care : meta-analysis in context
M. Egger (2001)
How important is publication bias? A synthesis of available data.
K. Dickersin (1997)
10.1046/J.1365-2753.2000.00236.X
Individual patient data meta-analysis of randomized anti-epileptic drug monotherapy trials.
P. Williamson (2000)
Publication bias in statistical overview of trials: example of psychological rehabilitation following myocardial infarction [abstract
RR West (1997)
INTERNATIONAL CONFERENCE ON HARMONISATION OF TECHNICAL REQUIREMENTS FOR REGISTRATION OF PHARMACEUTICALS FOR HUMAN USE ICH HARMONISED TRIPARTITE GUIDELINE
PHARMACEUTICAL DEVELOPMENT (1998)
[Data torturing].
A. Kohn (1994)
10.1136/bmj.316.7126.221
Meta-analysis: Unresolved issues and future developments
G. Smith (1998)
10.1016/0895-4356(92)90072-U
Bias in meta-analytic research.
D. Felson (1992)
10.1056/NEJM198708133170706
Statistical problems in the reporting of clinical trials. A survey of three medical journals.
S. Pocock (1987)
10.1046/J.1365-2753.2002.00314.X
Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee.
S. Hahn (2002)
10.1136/bmj.326.7400.1171
Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications
H. Melander (2003)
Meta-analysis of controlled clinical trials.
C. Naylor (1989)
10.1136/bmj.328.7430.22
Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
H. Soares (2004)
10.1001/JAMA.1990.03440100121018
Underreporting research is scientific misconduct.
I. Chalmers (1990)
Research governance must focus on research training
GD Murray (2001)
10.1016/0197-2456(89)90017-2
Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.
P. Gøtzsche (1989)
10.1186/1471-8219-2-4
Publishing study protocols: making them visible will improve registration, reporting and recruitment
F. Godlee (2001)
10.1056/NEJM199011083231921
Editorial review of protocols for clinical trials.
J P Siegel (1990)
10.7326/0003-4819-134-8-200104170-00012
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
D. Altman (2001)
10.1001/JAMA.287.21.2831
Publishing protocols of systematic reviews: comparing what was done to what was planned.
C. Silagy (2002)
10.1016/S0140-6736(98)07618-1
How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing
I. Chalmers (1999)
10.3310/HTA4100
Publication and related biases.
F. Song (2000)
Agreement on cooperation on clinical trials between physicians and the pharmaceutical industry
J P Siegel
10.1002/14651858.CD001319.PUB5
Colloid solutions for fluid resuscitation.
F. Bunn (2001)
10.1186/2048-4623-1-s3-pa005
Outcome reporting bias and individual patient data meta-analysis: a case study in surgery
K. McCormack (2001)
Changes between protocol and manuscript should be declared at submission
S. Goldbeck-Wood (2001)



This paper is referenced by
10.1016/j.jaci.2014.09.019
Rate of asthma trial outcomes reporting on ClinicalTrials.gov and in the published literature.
C. Stockmann (2014)
Registering Clinical Trial Results
D. Miller (2017)
10.1159/000318293
Efficacy and Effectiveness of Antidepressants: Current Status of Research
H. Pigott (2010)
10.1586/ers.11.84
Are we sure about the evidence for zinc in prophylaxis of the common cold?
J. Peters (2012)
Evaluation Report Early Start
D. Fergusson (2005)
10.3129/i09-113
The distortion of commercial research reported in the peer-reviewed literature.
Thomas J. Liesegang (2009)
10.1179/2047480612Z.00000000085
Implications of clinical trial data sharing for medical writers
J. Ross (2013)
10.1136/bmj.m982
Prospective registration and reporting of trial number in randomised clinical trials: global cross sectional study of the adoption of ICMJE and Declaration of Helsinki recommendations
Mustafa Al-Durra (2020)
10.1097/SLA.0b013e318283cf7f
Comparison of Randomized Controlled Trial Registry Entries and Content of Reports in Surgery Journals
R. Rosenthal (2013)
10.1136/bmj.d4002
Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
J. Sterne (2011)
10.1080/14653125.2017.1370773
Are Orthodontic Systematic Reviews Registered a Priori in PROSPERO?
S. Sideri (2017)
10.1093/ageing/afu086
Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012.
Eva Mann (2014)
10.1002/14651858.ED000127
Doug Altman's legacy to Cochrane and evidence synthesis.
J. Deeks (2018)
10.1200/JCO.2015.64.3437
Evolution of Randomized Trials in Advanced/Metastatic Soft Tissue Sarcoma: End Point Selection, Surrogacy, and Quality of Reporting.
A. Zer (2016)
Validity and Inter-Rater Reliability Testing of Quality Assessment Instruments
L. Hartling (2012)
10.1136/bmj.d7292
Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis
J. Ross (2012)
10.1038/jid.2011.337
The growth of clinical trials and systematic reviews in informing dermatological patient care.
H. Williams (2012)
10.1371/journal.pmed.1002046
Agreements between Industry and Academia on Publication Rights: A Retrospective Study of Protocols and Publications of Randomized Clinical Trials
B. Kasenda (2016)
10.1186/s12916-016-0639-x
Association between trial registration and treatment effect estimates: a meta-epidemiological study
A. Dechartres (2016)
10.1111/J.1365-2133.2005.06727.X
Compulsory registration of clinical trials
A. Ormerod (2005)
10.1016/J.ZEFQ.2010.03.029
[The relation between publication bias and clinical trials funding].
G. Schott (2010)
10.1007/s11892-015-0680-8
Mobile Phone and Smartphone Technologies for Diabetes Care and Self-Management
L. Garabedian (2015)
10.5694/j.1326-5377.2010.tb04081.x
The ABC breast cancer cluster: the bad news about a good outcome
M. Coory (2010)
Conflicts of interest in biomedical publishing
A. Lundh (2012)
10.1186/s12910-019-0405-7
The effects of industry funding and positive outcomes in the interpretation of clinical trial results: a randomized trial among Dutch psychiatrists
J. K. Tijdink (2019)
Análisis de la estética y la remodelación ósea en implantes post-extracción rehabilitados de manera inmediata con coronas unitarias atornilladas. Estudio clínico retrospectivo a 5 años
B. F. Carro (2017)
10.1007/s00264-009-0895-x
Reamed and unreamed intramedullary nailing for the treatment of open and closed tibial fractures: a subgroup analysis of randomised trials
Deting Xue (2009)
10.1097/IAE.0000000000001280
PARS PLANA VITRECTOMY FOR DIABETIC MACULAR EDEMA: A Systematic Review, Meta-Analysis, and Synthesis of Safety Literature
T. Jackson (2017)
10.1016/S0140-6736(06)68153-1
What are the implications of optimism bias in clinical research?
I. Chalmers (2006)
10.1016/j.medin.2017.07.008
Clinical research in critical care. Difficulties and perspectives.
J. Latour-Pérez (2018)
10.1016/J.MEDINE.2017.07.001
Clinical research in critical care. Difficulties and perspectives.
J. Latour-Pérez (2018)
10.1016/j.jclinepi.2017.07.010
Comparison between publicly accessible publications, registries, and protocols of phase III trials indicated persistence of selective outcome reporting.
S. Zhang (2017)
See more
Semantic Scholar Logo Some data provided by SemanticScholar