Online citations, reference lists, and bibliographies.
← Back to Search

Should Meta-analyses Of Interventions Include Observational Studies In Addition To Randomized Controlled Trials? A Critical Examination Of Underlying Principles.

I. Shrier, J. Boivin, R. Steele, R. Platt, Andrea Furlan, R. Kakuma, J. Brophy, M. Rossignol
Published 2007 · Medicine

Save to my Library
Download PDF
Analyze on Scholarcy Visualize in Litmaps
Share
Reduce the time it takes to create your bibliography by a factor of 10 by using the world’s favourite reference manager
Time to take this seriously.
Get Citationsy
Some authors argue that systematic reviews and meta-analyses of intervention studies should include only randomized controlled trials because the randomized controlled trial is a more valid study design for causal inference compared with the observational study design. However, a review of the principal elements underlying this claim (randomization removes the chance of confounding, and the double-blind process minimizes biases caused by the placebo effect) suggests that both classes of study designs have strengths and weaknesses, and including information from observational studies may improve the inference based on only randomized controlled trials. Furthermore, a review of empirical studies suggests that meta-analyses based on observational studies generally produce estimates of effect similar to those from meta-analyses based on randomized controlled trials. The authors found that the advantages of including both observational studies and randomized studies in a meta-analysis could outweigh the disadvantages in many situations and that observational studies should not be excluded a priori.
This paper references
10.1016/b978-0-12-809730-4.00024-0
Probability and Statistics
M. H. Degroot (1977)
10.1016/0002-9343(82)90815-4
Randomized versus historical controls for clinical trials.
H. Sacks (1982)
10.7326/0003-4819-102-6-874_2
Clinical epidemiology : a basic science for clinical medicine
D. Sackett (1985)
10.1002/SIM.4780040319
The need for randomization in the study of intended effects.
V. Siskind (1985)
10.1136/bmj.293.6543.363
Does informed consent influence therapeutic outcome? A clinical trial of the hypnotic activity of placebo in patients admitted to hospital.
R. Dahan (1986)
10.1016/0167-6296(86)90020-2
Measurement of health state utilities for economic appraisal.
G. Torrance (1986)
10.1002/SIM.4780070807
A note on graphical presentation of estimated odds ratios from several clinical trials.
R. Galbraith (1988)
10.1016/0140-6736(91)90201-Y
Publication bias in clinical research
P. Easterbrook (1991)
10.2307/2348741
Categorical Data Analysis.
D. Lendrem (1991)
Deciding on the best therapy
DL Sackett (1991)
10.1136/JECH.46.5.549
Clinical Epidemiology: A Basic Science for Clinical Medicine
R. Logan (1992)
A randomised clinical trial of the effect of informed consent on the analgesic activity of placebo and naproxen in cancer pain.
J. Bergmann (1994)
10.2307/2684909
Statistics: A Bayesian Perspective
D. Berry (1995)
10.1080/00031305.1996.10473555
Meta-Analysis and Response Surface Extrapolation: A Least Squares Approach
Wilfried R. Vanhonacker (1996)
10.1016/S0140-6736(97)10111-8
Powerful placebo: the dark side of the randomised controlled trial
T. Kaptchuk (1998)
10.1136/bmj.317.7167.1185
The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials
R. Kunz (1998)
10.1097/00042752-199910000-00007
Stretching before exercise does not reduce the risk of local muscle injury: a critical review of the clinical and basic science literature.
I. Shrier (1999)
10.1097/00001648-199901000-00005
Causal diagrams for epidemiologic research.
S. Greenland (1999)
10.1097/00005768-200002000-00004
A randomized trial of preexercise stretching for prevention of lower-limb injury.
R. Pope (2000)
10.1002/0470842547
Dictionary of Pharmacoepidemiology
B. Bégaud (2000)
10.1016/S0895-4356(99)00160-2
Randomized trials versus observational studies in adolescent pregnancy prevention.
G. Guyatt (2000)
10.5860/choice.47-3771
Causality: Models, Reasoning and Inference
J. Pearl (2000)
10.3310/HTA4340
A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies.
R. MacLehose (2000)
10.1056/NEJM200006223422507
Randomized, controlled trials, observational studies, and the hierarchy of research designs.
J. Concato (2000)
10.1056/NEJM200010193431613
Observational studies and randomized trials.
R. Kunz (2000)
The art and science of cause and effect. In: Causality: models, reasoning and inference
J. Pearl (2000)
Causality: models, reasoning and inference. Cambridge
J Pearl (2000)
The art and science of cause and effect. In: Causality: models, reasoning and inference. Cambridge
J Pearl (2000)
10.1016/S0895-4356(00)00347-4
The double-blind, randomized, placebo-controlled trial: gold standard or golden calf?
T. Kaptchuk (2001)
10.1002/9780470693926
Systematic Reviews in Health Care
M. Egger (2001)
10.1001/JAMA.286.7.821
Comparison of evidence of treatment effects in randomized and nonrandomized studies.
J. Ioannidis (2001)
1 Rationale, potentials, and promise of systematic reviews
M. Egger (2001)
Systematic reviews in health care : meta-analysis in context
M. Egger (2001)
10.1093/BIOSTATISTICS/2.4.463
On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions.
S. Greenland (2001)
10.1146/ANNUREV.PUBLHEALTH.22.1.189
Confounding in health research.
S. Greenland (2001)
10.1097/00006254-200110000-00021
Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment.
A. Hrõbjartsson (2001)
10.1136/bmj.325.7362.468
Effects of stretching before and after exercising on muscle soreness and risk of injury: systematic review
R. Herbert (2002)
10.1093/AJE/155.2.176
Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology.
M. Hernán (2002)
10.1001/JAMA.287.22.2973
Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials.
E. Balk (2002)
10.3310/HTA7270
Evaluating non-randomised intervention studies.
J. Deeks (2003)
10.1136/bjsm.37.6.473
Cochrane Reviews: new blocks on the kids
I. Shrier (2003)
10.1249/01.MSS.0000117134.83018.F7
The impact of stretching on sports injury risk: a systematic review of the literature.
S. Thacker (2004)
10.1097/01.ede.0000135174.63482.43
A Structural Approach to Selection Bias
M. Hernán (2004)
10.1136/jech.2002.006361
A definition of causal effect for epidemiological research
M. Hernán (2004)
10.1007/BF00589896
Effect of instructions and nicotine on smoking cessation, withdrawal symptoms and self-administration of nicotine gum
J. Hughes (2004)
10.1007/BF02244650
Do double-blind studies with informed consent yield externally valid results?
I. Kirsch (2005)
10.1016/J.ACCREVIEW.2005.09.021
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
J. Ioannidis (2005)
10.1111/J.0006-341X.2005.454_3.X
Discussion on "Statistical Issues Arising in the Women's Health Initiative"
N. Day (2005)
10.1111/J.0006-341X.2005.454_5.X
Discussion on "Statistical Issues Arising in the Women's Health Initiative"
D. Freedman (2005)
10.1016/J.ACCREVIEW.2005.05.034
The Effects of a Smoking Cessation Intervention on 14.5-Year Mortality
N. Anthonisen (2005)
10.1111/J.0006-341X.2005.454_1.X
Statistical issues arising in the Women's Health Initiative.
R. Prentice (2005)
Why randomized trials and non-randomized studies of the same interventions agree or disagree. Presented at the 4th Canadian Cochrane Symposium
Ad Furlan (2005)
Meta-regression of randomized and non-randomized studies of treatments for low-back pain
AD Furlan (2005)
Meta-regression of randomized and non-randomized studies of treatments for low-back pain. Presented at the 4th Canadian Cochrane Symposium
Ad Furlan (2005)
Why randomized trials and non-randomized studies of the same interventions agree or disagree
AD Furlan (2005)
Am J Epidemiol
Shrier (2007)
10.1002/9780470693926.CH9
Why and how Sources of Heterogeneity Should be Investigated
S. Thompson (2008)
10.1002/9780470693926.CH26
The Cochrane Collaboration in the 21st Century: Ten Challenges and One Reason Why They Must Be Met
A. Oxman (2008)
Agreement between randomized trials and non - randomized studies was influenced by methodological quality and homogeneity
AD Furlan
Agreement between randomized trials and nonrandomized studies was influenced by methodological quality and homogeneity
AD Furlan



This paper is referenced by
10.1016/J.ADOMS.2021.100014
Does additional intraoperative imaging improves outcomes in isolated zygomatic arch fractures: Systematic review
Ashutosh Kumar Singh (2021)
10.1097/MD.0000000000024124
Impact of surgical margin on survival in extremity soft tissue sarcoma
W. Jang (2021)
10.2196/33534
Addendum to the Acknowledgments: Stepwise-Hierarchical Pooled Analysis for Synergistic Interpretation of Meta-analyses Involving Randomized and Observational Studies: Methodology Development.
I. Shin (2021)
10.1016/j.jaac.2021.05.005
Systematic Review and Meta-analysis: Depressive Symptoms and Risky Behaviors Among Adolescents in Low- and Middle-Income Countries.
J. Pozuelo (2021)
10.1186/s12902-021-00699-5
Levothyroxine and the risk of adverse pregnancy outcomes in women with subclinical hypothyroidism: a systematic review and meta-analysis
M. Bein (2021)
10.1080/1744666X.2021.1908128
Efficacy and safety of Tocilizumab in severe and critical COVID-19: A Systematic Review and Meta-Analysis
S. Rezaei (2021)
10.3390/cancers13122912
Is the Concurrent Use of Sorafenib and External Radiotherapy Feasible for Advanced Hepatocellular Carcinoma? A Meta-Analysis
C. H. Rim (2021)
10.1186/s13643-021-01609-5
Effects of psychosocial interventions among people cared for in emergency departments after a suicide attempt: a systematic review protocol
Ana Paula Coutinho da Silva (2021)
10.1016/J.JCHIRV.2020.12.005
L’omentoplastie diminue l’infection du site opératoire profond comparée au drainage externe après chirurgie conservatrice de l’échinococcose kystique du foie : méta-analyse avec méta-régression
C. Dziri (2021)
10.47895/amp.vi0.3178
Utility of Ipsilateral Medial Fibular Transport Using the Ilizarov Frame in the Treatment for Non-elderly Patients Sustaining Massive Tibial Bone Defects as a Sequela of Trauma and Infection: A Systematic Review
Patricio E. Dumlao III (2021)
10.1007/s12325-021-01731-9
Preserved Versus Preservative-Free Latanoprost for the Treatment of Glaucoma and Ocular Hypertension: A Post Hoc Pooled Analysis
P. Harasymowycz (2021)
10.1111/ene.14839
Thrombolytic therapy for wake‐up stroke: A systematic review and meta‐analysis
Brian Mac Grory (2021)
10.2196/29642
Stepwise-hierarchical pooled analysis: a synergistic interpretation of meta-analysis including randomized and observational studies: A methodologic study.
I. Shin (2021)
10.1016/j.jviscsurg.2021.03.009
Omentoplasty decreases deep organ space surgical site infection compared with external tube drainage after conservative surgery for hepatic cystic echinococcosis: Meta-analysis with a meta-regression.
C. Dziri (2021)
10.1007/s00592-021-01738-2
Prevalence of periodontitis in people clinically diagnosed with diabetes mellitus: a meta-analysis of epidemiologic studies
Mengwen Zheng (2021)
10.1080/01621459.2018.1529598
Sensitivity Analysis for Unmeasured Confounding in Meta-Analyses
Maya B. Mathur (2020)
10.1016/j.chest.2020.06.078
Acetazolamide for Obstructive and Central Sleep Apnea: A Comprehensive Systematic Review and Meta-Analysis.
C. Schmickl (2020)
10.1159/000508415
Impact of Radiotherapy on Prognosis in Patients Diagnosed with Metastatic Prostate Cancer: A Systematic Review and Meta-Analysis
Shuai Liu (2020)
10.1097/j.jcrs.0000000000000228
Comparison of femtosecond laser-assisted cataract surgery and conventional cataract surgery: meta-analysis and systematic review.
Caroline M. Kolb (2020)
10.1016/j.bja.2020.10.022
Prospectively assessed neurodevelopmental outcomes in studies of anaesthetic neurotoxicity in children: a systematic review and meta-analysis.
C. Ing (2020)
10.1080/13548506.2020.1732431
A systematic review of the effectiveness of sleep hygiene in children with ADHD
J. Nikles (2020)
10.1186/s13012-020-01054-3
Effectiveness of interventions to improve rates of intravenous thrombolysis using behaviour change wheel functions: a systematic review and meta-analysis
Md Golam Hasnain (2020)
10.34250/jkccn.2020.13.1.63
Mobile Health Interventions for Community-Dwelling Patients with Heart Diseases: An Integrative Review
J. Ko (2020)
10.5125/jkaoms.2020.46.3.162
The impact of the alveolar bone sites on early implant failure: a systematic review with meta-analysis
A. Fouda (2020)
Piezoelectric bone surgery for implant site preparation compared with conventional drilling techniques: A systematic review, meta-analysis and trial sequential analysis.
C. Stacchi (2020)
10.21037/atm.2020.01.81
Benefits of adjuvant treatment including trastuzumab in HER2-positive pT1a-bN0M0 breast cancer: a systematic review and meta-analysis.
H. Lee (2020)
10.1186/s12911-020-01343-3
Can statistical adjustment guided by causal inference improve the accuracy of effect estimation? A simulation and empirical research based on meta-analyses of case–control studies
R. Yan (2020)
10.1097/PHM.0000000000001656
Letter to the Editor Regarding: "Effects of exercise in patients with amyotrophic lateral sclerosis: A systematic review and meta-analysis".
Cara Donohue (2020)
10.1111/apt.15925
Systematic review with meta-analysis: limited benefits from early colonoscopy in acute lower gastrointestinal bleeding.
O. Kherad (2020)
10.3390/cancers12082125
Oncologic Benefit of Adjuvant Chemoradiation after D2 Gastrectomy: A Stepwise Hierarchical Pooled Analysis and Systematic Review
C. H. Rim (2020)
10.1016/j.cmi.2020.10.036
Efficacy and safety of tocilizumab in COVID-19 patients: a living systematic review and meta-analysis
I. Tleyjeh (2020)
10.1002/hed.26264
Reirradiation with intensity‐modulated radiation therapy for recurrent or secondary head and neck cancer: Meta‐analysis and systematic review
Jeongshim Lee (2020)
See more
Semantic Scholar Logo Some data provided by SemanticScholar