Online citations, reference lists, and bibliographies.
← Back to Search

The Quality Of Reporting Methods And Results In Network Meta-Analyses: An Overview Of Reviews And Suggestions For Improvement

B. Hutton, G. Salanti, A. Chaimani, D. Caldwell, Chris Schmid, K. Thorlund, E. Mills, F. Catalá-López, Lucy Turner, D. Altman, D. Moher
Published 2014 · Biology, Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
Introduction Some have suggested the quality of reporting of network meta-analyses (a technique used to synthesize information to compare multiple interventions) is sub-optimal. We sought to review information addressing this claim. Objective To conduct an overview of existing evaluations of quality of reporting in network meta-analyses and indirect treatment comparisons, and to compile a list of topics which may require detailed reporting guidance to enhance future reporting quality. Methods An electronic search of Medline and the Cochrane Registry of methodologic studies (January 2004–August 2013) was performed by an information specialist. Studies describing findings from quality of reporting assessments were sought. Screening of abstracts and full texts was performed by two team members. Descriptors related to all aspects of reporting a network meta-analysis were summarized. Results We included eight reports exploring the quality of reporting of network meta-analyses. From past reviews, authors found several aspects of network meta-analyses were inadequately reported, including primary information about literature searching, study selection, and risk of bias evaluations; statement of the underlying assumptions for network meta-analysis, as well as efforts to verify their validity; details of statistical models used for analyses (including information for both Bayesian and Frequentist approaches); completeness of reporting of findings; and approaches for summarizing probability measures as additional important considerations. Conclusions While few studies were identified, several deficiencies in the current reporting of network meta-analyses were observed. These findings reinforce the need to develop reporting guidance for network meta-analyses. Findings from this review will be used to guide next steps in the development of reporting guidance for network meta-analysis in the format of an extension of the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) Statement.
This paper references
10.1016/j.jval.2011.01.011
Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2.
D. Hoaglin (2011)
10.2147/CLEP.S16526
Multiple treatment comparison meta-analyses: a step forward into complexity
E. Mills (2011)
Indirect Evidence: Indirect Treatment Comparisons in Meta-Analysis. Ottawa: Canadian Agency for Drugs and Technologies in Health
G Wells (2009)
10.1016/S0895-4356(97)00049-8
The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials.
H. Bucher (1997)
10.1016/j.jclinepi.2010.03.016
Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial.
G. Salanti (2011)
10.1016/j.jnn.2010.11.006
National Institute for Health and Clinical Excellence
A. Dhar (2005)
NICE DSU Technical Support Document 7: Evidence Synthesis of Treatment Efficacy in Decision Making: A Reviewer’s Checklist
A. Ades (2012)
10.1002/pst.1592
Guidance on the implementation and reporting of a drug safety Bayesian network meta-analysis.
D. Ohlssen (2014)
Indirect comparisons Methods and validity
10.1002/(SICI)1097-0258(19961230)15:24<2733::AID-SIM562>3.0.CO;2-0
Borrowing strength from external trials in a meta-analysis.
J. Higgins (1996)
10.1016/j.jclinepi.2013.07.014
Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009.
A. Lee (2014)
10.1371/journal.pmed.1000100
The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
A. Liberati (2009)
10.1002/jrsm.1037
Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool.
G. Salanti (2012)
10.1371/journal.pone.0011054
Indirect Comparisons: A Review of Reporting and Methodological Quality
S. Donegan (2010)
10.1016/J.JVAL.2011.02.752
PMS69 THE USE OF MIXED TREATMENT COMPARISONS IN NICE TECHNOLOGY APPRAISALS
C. Brooks-Rooney (2011)
10.1016/j.ijsu.2010.02.007
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
D. Moher (2010)
10.1136/bmjopen-2013-003111
Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review
D. Sobieraj (2013)
10.1371/journal.pone.0086754
Characteristics of Networks of Interventions: A Description of a Database of 186 Published Networks
A. Nikolakopoulou (2014)
10.1177/1355819613498379
Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons
S. Tan (2013)
10.1136/bmj.b1147
Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews
F. Song (2009)
Use of indirect comparison in HTA submissions The use of mixed treatment comparisons in NICE technology appraisals
F Buckley (2009)
NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials
S. Dias (2011)
10.1016/j.jclinepi.2008.10.012
An evidence-based practice guideline for the peer review of electronic search strategies.
M. Sampson (2009)
Used of Mixed Treatment Comparisons in Systematic Reviews. Methods Research report Prepared by the University of Connecticut/Hartford Hospital evidencebased practice center
C Coleman (2012)
10.1136/bmj.f3675
Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review
Aïda Bafeta (2013)
10.1002/SIM.1201
Network meta-analysis for indirect treatment comparisons.
T. Lumley (2002)
NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and biasadjustment
S Dias (2011)
NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and biasadjustment. Available from http://www nicedsu.org.uk
S Dias (2011)
10.3736/JCIM20090918
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
D. Moher (2009)
Comparators and Comparisons: Direct and Indirect Comparisons Available: https://www. cc-arcc.ca/common/pages/UserFile.aspx?fileId = 282143
(2008)
10.1136/bmj.b2700
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration
A. Liberati (2009)
10.1016/S1098-3015(10)73531-2
PHP82 USE OF INDIRECT COMPARISON IN HTA SUBMISSIONS
F. Buckley (2009)
Use of Mixed Treatment Comparisons in Systematic Reviews
C. Coleman (2012)
Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment
S. Dias (2011)
10.2165/00019053-200826090-00006
Use of Indirect and Mixed Treatment Comparisons for Technology Assessment
A. Sutton (2012)
10.1177/0962280207080643
Evaluation of networks of randomized trials
G. Salanti (2008)
10.1001/2012.JAMA.11228
How to use an article reporting a multiple treatment comparison meta-analysis.
E. Mills (2012)
Comparators and Comparisons: Direct and Indirect Comparisons
(2013)
10.1177/0962280213500185
A Bayesian network meta-analysis for binary outcome: how to do it
T. Greco (2016)
NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials.
S. Dias (2011)
10.1016/J.JVAL.2011.02.126
PHP56 DEMONSTRATING CLINICAL-EFFECTIVENESS USING INDIRECT AND MIXED TREATMENT COMPARISON ANALYSIS: A REVIEW OF MANUFACTURERS' SINGLE TECHNOLOGY APPRAISAL (STA) SUBMISSIONS TO THE NATIONAL INSTITUTE FOR HEALTH AND CLINICAL EXCELLENCE (NICE)
M. Bending (2011)
10.1136/bmj.331.7521.897
Simultaneous comparison of multiple treatments: combining direct and indirect evidence
D. Caldwell (2005)
10.1186/1741-7015-9-79
Network meta-analysis-highly attractive but more methodological research is needed
Tianjing Li (2011)
10.1503/cmaj.081086
Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses
J. Ioannidis (2009)
Indirect Comparisons: Methods and Validity. Reporting Quality of Network Meta-Analyses PLOS ONE | www
Haute Autorite De Sante (2009)
10.1002/SIM.1875
Combination of direct and indirect evidence in mixed treatment comparisons.
G. Lu (2004)
10.1016/j.jval.2011.04.002
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1.
J. Jansen (2011)



This paper is referenced by
Updating Quantitative Synthesis
(2016)
10.12688/gatesopenres.13082.1
Interventions to improve linear growth during exclusive breastfeeding life-stage for children aged 0-6 months living in low- and middle-income countries: a systematic review and network and pairwise meta-analyses.
Jay J. H. Park (2019)
[Network meta-analysis: mixed and indirect treatment comparisons. a new method to the service of clinical epidemiology and public health].
H. García-Perdomo (2016)
10.1136/bmjopen-2018-022769
Reporting guidelines for health research: protocol for a cross-sectional analysis of the EQUATOR Network Library
F. Catalá-López (2019)
10.18549/PharmPract.2017.01.943
Network meta-analysis: a technique to gather evidence from direct and indirect comparisons
F. S. Tonin (2017)
DECLARATIONOFTRANSPARENCY:PROMOTINGAMORECOMPLETE, HONESTANDADEQUATEPUBLICATIONOFSCIENTIFICARTICLES
Ferrán Catalá-López (2014)
10.1007/978-3-319-25655-9_18
Case Study in Orthopedics
Maciej Płaszewski (2016)
10.1371/journal.pone.0113277
What Guidance Are Researchers Given on How to Present Network Meta-Analyses to End-Users such as Policymakers and Clinicians? A Systematic Review
S. M. Sullivan (2014)
10.1016/j.ijrobp.2018.09.037
Efficacy and Safety Evaluation of the Various Therapeutic Options in Locally Advanced Cervix Cancer: A Systematic Review and Network Meta-Analysis of Randomized Clinical Trials.
N. R. Datta (2019)
10.1093/ije/dyy197
Methodological quality assessment of network meta-analysis of drug interventions: implications from a systematic review.
F. S. Tonin (2018)
Work Package 4 Methodological guidance , recommendations and illustrative case studies for ( network ) meta-analysis and modelling to predict real-world effectiveness using individual participant and / or aggregate data
Noemi Hummela (2017)
10.1371/journal.pone.0131953
Correction: An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review
J. Chambers (2015)
10.1136/bmjopen-2017-015888
Impact of heterogeneity and effect size on the estimation of the optimal information size: analysis of recently published meta-analyses
J. Garcia-Alamino (2017)
10.1186/s12916-017-0832-6
Erratum to: Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review
W. Zarin (2017)
10.1111/jebm.12264
Software and package applicating for network meta‐analysis: A usage‐based comparative study
C. Xu (2018)
10.1016/j.ajodo.2015.12.025
Comparative effectiveness of pharmacologic and nonpharmacologic interventions for orthodontic pain relief at peak pain intensity: A Bayesian network meta-analysis.
S. S. Sandhu (2016)
10.1038/s41598-019-40951-6
Upfront Surgery versus Neoadjuvant Therapy for Resectable Pancreatic Cancer: Systematic Review and Bayesian Network Meta-analysis
A. Bradley (2019)
10.1371/journal.pone.0196644
Mapping the characteristics of network meta-analyses on drug therapy: A systematic review
F. S. Tonin (2018)
10.1007/s00167-014-3311-z
Patellofemoral resurfacing and patellar denervation in primary total knee arthroplasty
A. Arirachakaran (2014)
10.1016/j.jclinepi.2015.05.027
Publication bias and small-study effects magnified effectiveness of antipsychotics but their relative ranking remained invariant.
D. Mavridis (2016)
10.1186/s12874-019-0829-2
BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses
Audrey Béliveau (2019)
10.1371/journal.pone.0121715
An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review
J. Chambers (2015)
10.5114/amsad.2020.95884
Usefulness of resveratrol supplementation in decreasing cardiometabolic risk factors comparing subjects with metabolic syndrome and healthy subjects with or without obesity: meta-analysis using multinational, randomised, controlled trials
C. Sergi (2020)
10.1016/j.medcli.2016.02.025
[The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA].
B. Hutton (2016)
Meta-analytic approaches for summarising and comparing the accuracy of medical tests
Yemisi Takwoingi (2016)
10.1371/journal.pone.0199540
Influence of cast change interval in the Ponseti method: A systematic review
R. B. Giesberts (2018)
10.1097/MD.0000000000004529
Comparative efficacy of interventions on nonalcoholic fatty liver disease (NAFLD)
R. Sawangjit (2016)
10.1136/bmj.k4029
Efficacy, effectiveness, and safety of herpes zoster vaccines in adults aged 50 and older: systematic review and network meta-analysis
A. Tricco (2018)
10.1016/j.spinee.2015.05.018
Strategies to improve the credibility of meta-analyses in spine surgery: a systematic survey.
N. Evaniew (2015)
10.5195/jmla.2018.82
Roles for librarians in systematic reviews: a scoping review
A. Spencer (2018)
DECLARACIÓNDETRANSPARENCIA:PROMOVIENDOUNAPUBLICACIÓN MÁSCOMPLETA,HONESTAYADECUADADELOSARTÍCULOSCIENTÍFICOS
Ferrán Catalá-López (2014)
10.1186/s12916-016-0764-6
Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review
W. Zarin (2017)
See more
Semantic Scholar Logo Some data provided by SemanticScholar