Online citations, reference lists, and bibliographies.
← Back to Search

A GRADE Working Group Approach For Rating The Quality Of Treatment Effect Estimates From Network Meta-analysis

M. Puhan, H. Schünemann, M. H. Murad, Tianjing Li, R. Brignardello‐Petersen, J. Singh, A. Kessels, G. Guyatt
Published 2014 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
This paper references
Evidence Synthesis for Decision Making 6: Embedding Evidence Synthesis in Probabilistic Cost-effectiveness Analysis
S. Dias (2013)
10.1001/2012.JAMA.11228
How to use an article reporting a multiple treatment comparison meta-analysis.
E. Mills (2012)
†Contributing direct evidence of moderate quality. ‡Indirectness. §Incoherence. ¶Limitations (risk of bias)
Imprecision
10.1002/SIM.1875
Combination of direct and indirect evidence in mixed treatment comparisons.
G. Lu (2004)
10.1016/S0140-6736(07)61444-5
Outcomes associated with drug-eluting and bare-metal stents: a collaborative network meta-analysis
C. Stettler (2007)
10.1016/j.ahj.2013.03.011
Risk of stroke with percutaneous coronary intervention compared with on-pump and off-pump coronary artery bypass graft surgery: Evidence from a comprehensive network meta-analysis.
T. Palmerini (2013)
10.1016/j.jclinepi.2010.09.011
GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology.
G. Guyatt (2011)
10.1371/journal.pone.0092508
The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement
B. Hutton (2014)
10.1016/j.jval.2011.01.011
Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2.
D. Hoaglin (2011)
10.1210/jc.2011-3060
Clinical review. Comparative effectiveness of drug treatments to prevent fragility fractures: a systematic review and network meta-analysis.
M. H. Murad (2012)
10.1136/heartjnl-2013-303811
Remote monitoring after recent hospital discharge in patients with heart failure: a systematic review and network meta-analysis
A. Pandor (2013)
10.1002/jrsm.1044
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies‡
J. Higgins (2012)
10.1016/j.jclinepi.2011.01.012
GRADE guidelines 6. Rating the quality of evidence--imprecision.
G. Guyatt (2011)
10.1016/j.jclinepi.2013.11.006
Novel presentational approaches were developed for reporting network meta-analysis.
S. Tan (2014)
10.7326/0003-4819-159-2-201307160-00008
Conceptual and Technical Challenges in Network Meta-analysis
A. Cipriani (2013)
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
(2014)
10.1177/0962280207080643
Evaluation of networks of randomized trials
G. Salanti (2008)
10.1016/j.zefq.2012.10.019
[GRADE guidelines: 8. Rating the quality of evidence - indirectness].
A. Rasch (2012)
10.2307/j.ctv105b8xz.40
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Accepted : ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
Veit Goder (1978)
10.1016/j.zefq.2012.10.018
[GRADE guidelines: 7. Rating the quality of evidence - inconsistency].
M. Perleth (2012)
10.1093/ije/dys222
Evaluation of inconsistency in networks of interventions.
A. Veroniki (2013)
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe
10.6142/VGHN.33.2.218
Vitamin D and vitamin D analogues for preventing fractures in post-menopausal women and older men
Anne-Marie Hanff (2016)
10.1016/j.jclinepi.2010.07.015
GRADE guidelines: 3. Rating the quality of evidence.
Howard Balshem (2011)
10.1016/j.jclinepi.2010.07.017
GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).
G. Guyatt (2011)
10.1016/j.jval.2011.04.002
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1.
J. Jansen (2011)
10.1186/1741-7015-7-2
Inhaled drugs to reduce exacerbations in patients with chronic obstructive pulmonary disease: a network meta-analysis
M. Puhan (2009)
10.1016/j.jclinepi.2013.02.004
The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses.
Reem A Mustafa (2013)
10.1002/SIM.1201
Network meta-analysis for indirect treatment comparisons.
T. Lumley (2002)
10.13172/2053-079X-1-1-690
A simple regression model for network meta-analysis
A. Kessels (2013)
10.1016/j.jval.2014.01.004
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report.
J. Jansen (2014)
10.1186/1741-7015-11-159
Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers
J. Jansen (2013)
10.1371/journal.pone.0099682
Evaluating the Quality of Evidence from a Network Meta-Analysis
G. Salanti (2014)
10.1002/sim.3767
Checking consistency in mixed treatment comparison meta-analysis.
S. Dias (2010)
Cite this...
10.1177/0272989X08330120
How Far Do You Go? Efficient Searching for Indirect Evidence
N. Hawkins (2009)
10.1016/j.jclinepi.2011.01.011
GRADE guidelines: 5. Rating the quality of evidence--publication bias.
G. Guyatt (2011)
10.1186/1741-7015-9-79
Network meta-analysis-highly attractive but more methodological research is needed
Tianjing Li (2011)
10.1371/journal.pone.0076654
Graphical Tools for Network Meta-Analysis in STATA
A. Chaimani (2013)
10.1136/bmj.g4605
Revascularisation versus medical treatment in patients with stable coronary artery disease: network meta-analysis
Collet Jp (2014)
10.1016/S0140-6736(09)60046-5
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis
A. Cipriani (2009)
10.1136/bmj.b1147
Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews
F. Song (2009)



This paper is referenced by
10.3389/fphar.2019.01221
Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis
Clare Lee (2019)
Comparative ef fi cacy of non-pharmacological adjuvant therapies for quality of life in the patients with breast cancer receiving chemo-or radiotherapy A protocol for systematic review and Bayesian network meta-analysis
Zhiyun He (2018)
10.1080/19439342.2019.1691627
Deworming children for soil-transmitted helminths in low and middle-income countries: systematic review and individual participant data network meta-analysis
Vivian Welch (2019)
10.1016/j.cjca.2020.02.072
Cardiac Rehabilitation Programs for Chronic Heart Disease: A Bayesian Network Meta-analysis.
Rongzhong Huang (2020)
10.1186/s13643-017-0522-7
Management of frailty: a protocol of a network meta-analysis of randomized controlled trials
A. Negm (2017)
10.1016/S2468-1253(16)30157-1
Comparative efficacy of pharmacological strategies for management of type 1 hepatorenal syndrome: a systematic review and network meta-analysis.
A. Facciorusso (2017)
AssociationofNoninvasiveVentilationStrategiesWithMortality andBronchopulmonaryDysplasiaAmongPreterm Infants A Systematic Review andMeta-analysis
T. Isayama (2016)
10.1136/bmjopen-2015-009998
Effect of serotonin modulating pharmacotherapies on body mass index and dysglycaemia among children and adolescents: a systematic review and network meta-analysis protocol
Reem A Al Khalifah (2016)
10.1002/14651858.CD013261.pub2
Treatment of newly diagnosed glioblastoma in the elderly: a network meta-analysis.
C. Hanna (2020)
10.1016/j.jaac.2016.12.013
Acute Antipsychotic Treatment of Children and Adolescents With Schizophrenia-Spectrum Disorders: A Systematic Review and Network Meta-Analysis.
A. K. Pagsberg (2017)
10.1186/s13643-018-0754-1
Neoadjuvant chemotherapy regimens in treatment of breast cancer: a systematic review and network meta-analysis protocol
Mona Pathak (2018)
10.1016/j.cgh.2018.04.041
Should Distal Attachment Devices be Routinely Added for Colonoscopy?
Eelco C Brand (2018)
A protocol for a multiple treatment comparison meta-analysis of randomized controlled trials
Yasir Rehman (2019)
10.1001/jamaoncol.2018.6720
Therapeutic Options for Neuroendocrine Tumors: A Systematic Review and Network Meta-analysis
R. Kaderli (2019)
10.18632/oncotarget.15580
Systematic review with network meta-analysis: comparative efficacy of different enteral immunonutrition formulas in patients underwent gastrectomy
Guo-Min Song (2017)
10.1177/1533033820943241
Systematic Review and Meta-Analysis of Multitargeted Tyrosine Kinase Inhibitors in Patients With Intractable Metastatic Colorectal Cancer
Zhenzhen Gao (2020)
10.1136/bmj.k585
Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study
A. Nikolakopoulou (2018)
10.1097/MD.0000000000019403
Comparison of the efficacy and safety of shockwave lithotripsy, retrograde intrarenal surgery, percutaneous nephrolithotomy, and minimally invasive percutaneous nephrolithotomy for lower-pole renal stones
Sheng-Han Tsai (2020)
10.1111/joor.12539
Pharmacological treatment of oro‐facial pain – health technology assessment including a systematic review with network meta‐analysis
B. Häggman-Henrikson (2017)
A protocol for systematic review and network meta-analysis
Shuqing Shi (2020)
10.1038/s41598-019-57370-2
Comparative efficacy and safety of warfarin care bundles and novel oral anticoagulants in patients with atrial fibrillation: a systematic review and network meta-analysis
Siok Shen Ng (2020)
10.1097/MD.0000000000012321
The effectiveness of non-invasive brain stimulation on arousal and alertness in patients in coma or persistent vegetative state after traumatic brain injury
Y. Li (2018)
10.1016/S2468-1253(18)30231-0
Comparative efficacy and tolerability of pharmacological agents for management of mild to moderate ulcerative colitis: a systematic review and network meta-analyses.
N. Nguyen (2018)
Protocol for a systematic review and network meta-analysis
Saurabh Gupta (2019)
10.1016/j.jclinepi.2019.12.020
GRADE guidelines: 21 part 1. Study design, risk of bias and indirectness in rating the certainty across a body of evidence for test accuracy.
H. Schünemann (2020)
10.1016/j.zefq.2019.11.003
GRADE-Leitlinien: 18. Wie ROBINS-I und andere Instrumente zur Einschätzung des Risikos für Bias von nicht-randomisierten Studien verwendet werden sollten, um die Vertrauenswürdigkeit eines Evidenzkörpers zu bewerten
Johannes Morche (2020)
10.1007/s10067-019-04704-0
The effectiveness of treatments for Kashin–Beck disease: a systematic review and network meta-analysis
Kun Zou (2019)
10.3892/etm.2020.8494
Neoadjuvant treatments for resectable rectal cancer: A network meta-analysis
Wei Zhong (2020)
10.7150/thno.21815
The Changing Therapeutic Role of Chemo-radiotherapy for Loco-regionally Advanced Nasopharyngeal Carcinoma from Two/Three-Dimensional Radiotherapy to Intensity-Modulated Radiotherapy: A Network Meta-Analysis
R. You (2017)
10.1038/s41390-020-1005-4
T-piece resuscitator or self-inflating bag during neonatal resuscitation: a scoping review
Charles Christoph Roehr (2020)
10.1002/14651858.CD013252.PUB2
Non-vitamin K antagonist oral anticoagulants (NOACs) post-percutaneous coronary intervention: a network meta-analysis.
Samer Al Said (2019)
10.1111/iwj.13224
Feasibility and safety of image‐guided vacuum‐assisted breast biopsy: A PRISMA‐compliant systematic review and meta‐analysis of 20 000 population from 36 longitudinal studies
M. Fang (2019)
See more
Semantic Scholar Logo Some data provided by SemanticScholar