Online citations, reference lists, and bibliographies.
← Back to Search

A GRADE Working Group Approach For Rating The Quality Of Treatment Effect Estimates From Network Meta-analysis

M. Puhan, H. Schünemann, M. H. Murad, Tianjing Li, R. Brignardello‐Petersen, J. Singh, A. Kessels, G. Guyatt
Published 2014 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
This paper references
Evidence Synthesis for Decision Making 6: Embedding Evidence Synthesis in Probabilistic Cost-effectiveness Analysis
S. Dias (2013)
How to use an article reporting a multiple treatment comparison meta-analysis.
E. Mills (2012)
†Contributing direct evidence of moderate quality. ‡Indirectness. §Incoherence. ¶Limitations (risk of bias)
Combination of direct and indirect evidence in mixed treatment comparisons.
G. Lu (2004)
Outcomes associated with drug-eluting and bare-metal stents: a collaborative network meta-analysis
C. Stettler (2007)
Risk of stroke with percutaneous coronary intervention compared with on-pump and off-pump coronary artery bypass graft surgery: Evidence from a comprehensive network meta-analysis.
T. Palmerini (2013)
GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology.
G. Guyatt (2011)
The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement
B. Hutton (2014)
Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2.
D. Hoaglin (2011)
Clinical review. Comparative effectiveness of drug treatments to prevent fragility fractures: a systematic review and network meta-analysis.
M. H. Murad (2012)
Remote monitoring after recent hospital discharge in patients with heart failure: a systematic review and network meta-analysis
A. Pandor (2013)
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies‡
J. Higgins (2012)
GRADE guidelines 6. Rating the quality of evidence--imprecision.
G. Guyatt (2011)
Novel presentational approaches were developed for reporting network meta-analysis.
S. Tan (2014)
Conceptual and Technical Challenges in Network Meta-analysis
A. Cipriani (2013)
For personal use only: See rights and reprints Subscribe:
Evaluation of networks of randomized trials
G. Salanti (2008)
[GRADE guidelines: 8. Rating the quality of evidence - indirectness].
A. Rasch (2012)
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Accepted : ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
Veit Goder (1978)
[GRADE guidelines: 7. Rating the quality of evidence - inconsistency].
M. Perleth (2012)
Evaluation of inconsistency in networks of interventions.
A. Veroniki (2013)
For personal use only: See rights and reprints Subscribe
Vitamin D and vitamin D analogues for preventing fractures in post-menopausal women and older men
Anne-Marie Hanff (2016)
GRADE guidelines: 3. Rating the quality of evidence.
Howard Balshem (2011)
GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).
G. Guyatt (2011)
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1.
J. Jansen (2011)
Inhaled drugs to reduce exacerbations in patients with chronic obstructive pulmonary disease: a network meta-analysis
M. Puhan (2009)
The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses.
Reem A Mustafa (2013)
Network meta-analysis for indirect treatment comparisons.
T. Lumley (2002)
A simple regression model for network meta-analysis
A. Kessels (2013)
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report.
J. Jansen (2014)
Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers
J. Jansen (2013)
Evaluating the Quality of Evidence from a Network Meta-Analysis
G. Salanti (2014)
Checking consistency in mixed treatment comparison meta-analysis.
S. Dias (2010)
Cite this...
How Far Do You Go? Efficient Searching for Indirect Evidence
N. Hawkins (2009)
GRADE guidelines: 5. Rating the quality of evidence--publication bias.
G. Guyatt (2011)
Network meta-analysis-highly attractive but more methodological research is needed
Tianjing Li (2011)
Graphical Tools for Network Meta-Analysis in STATA
A. Chaimani (2013)
Revascularisation versus medical treatment in patients with stable coronary artery disease: network meta-analysis
Collet Jp (2014)
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis
A. Cipriani (2009)
Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews
F. Song (2009)

This paper is referenced by
Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis
Clare Lee (2019)
Comparative ef fi cacy of non-pharmacological adjuvant therapies for quality of life in the patients with breast cancer receiving chemo-or radiotherapy A protocol for systematic review and Bayesian network meta-analysis
Zhiyun He (2018)
Deworming children for soil-transmitted helminths in low and middle-income countries: systematic review and individual participant data network meta-analysis
Vivian Welch (2019)
Cardiac Rehabilitation Programs for Chronic Heart Disease: A Bayesian Network Meta-analysis.
Rongzhong Huang (2020)
Management of frailty: a protocol of a network meta-analysis of randomized controlled trials
A. Negm (2017)
Comparative efficacy of pharmacological strategies for management of type 1 hepatorenal syndrome: a systematic review and network meta-analysis.
A. Facciorusso (2017)
AssociationofNoninvasiveVentilationStrategiesWithMortality andBronchopulmonaryDysplasiaAmongPreterm Infants A Systematic Review andMeta-analysis
T. Isayama (2016)
Effect of serotonin modulating pharmacotherapies on body mass index and dysglycaemia among children and adolescents: a systematic review and network meta-analysis protocol
Reem A Al Khalifah (2016)
Treatment of newly diagnosed glioblastoma in the elderly: a network meta-analysis.
C. Hanna (2020)
Acute Antipsychotic Treatment of Children and Adolescents With Schizophrenia-Spectrum Disorders: A Systematic Review and Network Meta-Analysis.
A. K. Pagsberg (2017)
Neoadjuvant chemotherapy regimens in treatment of breast cancer: a systematic review and network meta-analysis protocol
Mona Pathak (2018)
Should Distal Attachment Devices be Routinely Added for Colonoscopy?
Eelco C Brand (2018)
A protocol for a multiple treatment comparison meta-analysis of randomized controlled trials
Yasir Rehman (2019)
Therapeutic Options for Neuroendocrine Tumors: A Systematic Review and Network Meta-analysis
R. Kaderli (2019)
Systematic review with network meta-analysis: comparative efficacy of different enteral immunonutrition formulas in patients underwent gastrectomy
Guo-Min Song (2017)
Systematic Review and Meta-Analysis of Multitargeted Tyrosine Kinase Inhibitors in Patients With Intractable Metastatic Colorectal Cancer
Zhenzhen Gao (2020)
Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study
A. Nikolakopoulou (2018)
Comparison of the efficacy and safety of shockwave lithotripsy, retrograde intrarenal surgery, percutaneous nephrolithotomy, and minimally invasive percutaneous nephrolithotomy for lower-pole renal stones
Sheng-Han Tsai (2020)
Pharmacological treatment of oro‐facial pain – health technology assessment including a systematic review with network meta‐analysis
B. Häggman-Henrikson (2017)
A protocol for systematic review and network meta-analysis
Shuqing Shi (2020)
Comparative efficacy and safety of warfarin care bundles and novel oral anticoagulants in patients with atrial fibrillation: a systematic review and network meta-analysis
Siok Shen Ng (2020)
The effectiveness of non-invasive brain stimulation on arousal and alertness in patients in coma or persistent vegetative state after traumatic brain injury
Y. Li (2018)
Comparative efficacy and tolerability of pharmacological agents for management of mild to moderate ulcerative colitis: a systematic review and network meta-analyses.
N. Nguyen (2018)
Protocol for a systematic review and network meta-analysis
Saurabh Gupta (2019)
GRADE guidelines: 21 part 1. Study design, risk of bias and indirectness in rating the certainty across a body of evidence for test accuracy.
H. Schünemann (2020)
GRADE-Leitlinien: 18. Wie ROBINS-I und andere Instrumente zur Einschätzung des Risikos für Bias von nicht-randomisierten Studien verwendet werden sollten, um die Vertrauenswürdigkeit eines Evidenzkörpers zu bewerten
Johannes Morche (2020)
The effectiveness of treatments for Kashin–Beck disease: a systematic review and network meta-analysis
Kun Zou (2019)
Neoadjuvant treatments for resectable rectal cancer: A network meta-analysis
Wei Zhong (2020)
The Changing Therapeutic Role of Chemo-radiotherapy for Loco-regionally Advanced Nasopharyngeal Carcinoma from Two/Three-Dimensional Radiotherapy to Intensity-Modulated Radiotherapy: A Network Meta-Analysis
R. You (2017)
T-piece resuscitator or self-inflating bag during neonatal resuscitation: a scoping review
Charles Christoph Roehr (2020)
Non-vitamin K antagonist oral anticoagulants (NOACs) post-percutaneous coronary intervention: a network meta-analysis.
Samer Al Said (2019)
Feasibility and safety of image‐guided vacuum‐assisted breast biopsy: A PRISMA‐compliant systematic review and meta‐analysis of 20 000 population from 36 longitudinal studies
M. Fang (2019)
See more
Semantic Scholar Logo Some data provided by SemanticScholar