Online citations, reference lists, and bibliographies.

The Feasibility Of Creating A Checklist For The Assessment Of The Methodological Quality Both Of Randomised And Non-randomised Studies Of Health Care Interventions.

S. H. Downs, N. Black
Published 1998 · Medicine
Referenced 1 time by Citationsy Users

Cite This
Download PDF
Analyze on Scholarcy
Share
OBJECTIVE: To test the feasibility of creating a valid and reliable checklist with the following features: appropriate for assessing both randomised and non-randomised studies; provision of both an overall score for study quality and a profile of scores not only for the quality of reporting, internal validity (bias and confounding) and power, but also for external validity. DESIGN: A pilot version was first developed, based on epidemiological principles, reviews, and existing checklists for randomised studies. Face and content validity were assessed by three experienced reviewers and reliability was determined using two raters assessing 10 randomised and 10 non-randomised studies. Using different raters, the checklist was revised and tested for internal consistency (Kuder-Richardson 20), test-retest and inter-rater reliability (Spearman correlation coefficient and sign rank test; kappa statistics), criterion validity, and respondent burden. MAIN RESULTS: The performance of the checklist improved considerably after revision of a pilot version. The Quality Index had high internal consistency (KR-20: 0.89) as did the subscales apart from external validity (KR-20: 0.54). Test-retest (r 0.88) and inter-rater (r 0.75) reliability of the Quality Index were good. Reliability of the subscales varied from good (bias) to poor (external validity). The Quality Index correlated highly with an existing, established instrument for assessing randomised studies (r 0.90). There was little difference between its performance with non-randomised and with randomised studies. Raters took about 20 minutes to assess each paper (range 10 to 45 minutes). CONCLUSIONS: This study has shown that it is feasible to develop a checklist that can be used to assess the methodological quality not only of randomised controlled trials but also non-randomised studies. It has also shown that it is possible to produce a checklist that provides a profile of the paper, alerting reviewers to its particular methodological strengths and weaknesses. Further work is required to improve the checklist and the training of raters in the assessment of external validity.
This paper references
10.1016/0197-2456(94)90023-X
Generalizing from clinical trials.
C. E. Davis (1994)
10.1001/jama.1994.03520240054041
A proposal for structured reporting of randomized controlled trials. The Standards of Reporting Trials Group.
E. Andrew (1994)
10.1046/j.1464-410X.1996.01422.x
The effectiveness of surgery for stress incontinence in women: a systematic review.
N. Black (1996)
10.1016/0197-2456(94)00031-W
Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists.
D. Moher (1995)
10.1136/bmj.310.6987.1122
Evidence based medicine: an approach to clinical problem-solving
W. Rosenberg (1995)
Generalising from clinical trials
C E Davis (1994)
Evaluating the e V ects of care during pregnancy and childbirth
I Chalmers
Methodological standards for controlled clinical trials of early contact and maternalinfant behaviour
M E Thomson (1984)
10.2307/2529310
The measurement of observer agreement for categorical data.
J. Landis (1977)
Bias in analytical research
D L Sackett (1979)
10.1016/0197-2456(94)90024-8
Generalizing the results of randomized clinical trials.
K. Bailey (1994)
10.1136/bmj.300.6736.1372
Sociodemographic and motivational characteristics of parents who volunteer their children for clinical research: a controlled study.
S. Harth (1990)
Methodological standards for controlled clinical trials of early contact and maternal - infant behaviour
ME Thomson (1984)
A systematic review of the effectiveness and safety of laparoscopic cholecystectomy.
Widdison Al (1996)
The e V ectiveness of surgery for stress incontinence in women : a systematic review
NA Black (1996)
10.1097/00043764-199603000-00007
Empirical Evidence of Bias. Dimensions of Methodological Quality Associated with Estimates of Treatment Effects in Controlled Trials
Igor Sartori (1996)
10.1016/S0140-6736(05)63833-0
Meta-analysis of randomised controlled trials
G. Smith (1997)
Evaluating the eVects of care during pregnancy and childbirth
I Chalmers (1989)
10.1001/jama.273.5.408
Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
K. Schulz (1995)
Instrument review criteria
GF Kuder (1995)
Generalising the results of randomized clinical trials
K R Bailey (1994)
Funding: no specific funding
10.1056/NEJM198206033062204
Reporting on methods in clinical trials.
R. Dersimonian (1982)
10.1002/sim.4780080408
How study design affects outcomes in comparisons of therapy. I: Medical.
G. Colditz (1989)
10.1136/bmj.3.5775.637-b
Ischaemic colitis without predisposing cause.
H. Dudley (1971)
An assessment of research methods reported in 103 scientific articles from two Canadian medical journals.
R. Badgley (1961)
10.1016/S0140-6736(05)63832-9
Meta-analysis of randomised controlled trials
P. Rothwell (1997)
10.1016/S0140-6736(05)64769-1
Clinical trial reporting
R. Peto (1996)
Methodologic standards for controlled clinical trials of early contact and maternal-infant behavior.
M. E. Thomson (1984)
10.1136/bmj.292.6523.810
Use of check lists in assessing the statistical content of medical studies.
M. J. Gardner (1986)
Standards of Reporting Trials Group. A proposal for structured reporting of randomised controlled trails
(1994)
10.1136/bmj.3.5723.637
Assessing Reports of Therapeutic Trials
N. Lionel (1970)
10.1056/NEJM198702193160806
Meta-analyses of randomized controlled trials.
H. Sacks (1987)
10.1007/BF02288391
The theory of the estimation of test reliability
G. F. Kuder (1937)
A proposal for structured reporting of randomised controlled trails
Landis (1994)
The eVectiveness of surgery for stress incontinence in women: a systematic review
N A Black (1996)
10.1016/0021-9681(79)90012-2
Bias in analytic research.
D. Sackett (1979)
10.1016/S0140-6736(96)90145-2
Results of a breast-cancer-surgery trial compared with observational data from routine practice
E. Marubini (1996)



This paper is referenced by
10.1590/S1413-81232013001100023
Deficiência de micronutrientes e crescimento linear: revisão sistemática de estudos observacionais
D. Pedraza. (2013)
10.1016/J.ACALIB.2007.03.006
The Efficacy of Computer-Assisted Instruction versus Face-to-Face Instruction in Academic Libraries: A Systematic Review
L. Zhang (2007)
10.1016/j.pmrj.2018.09.031
The Effects of Hydrotherapy on Balance, Functional Mobility, Motor Status, and Quality of Life in Patients with Parkinson Disease: A Systematic Review and Meta‐analysis
C. Pinto (2018)
10.26633/rpsp.2018.92
The associations that income, education, and ethnicity have with birthweight and prematurity: how close are they?
Ana Daniela Izoton de Sadovsky (2018)
10.1016/J.JCBS.2015.10.003
In search of the person in pain: A systematic review of conceptualization, assessment methods, and evidence for self and identity in chronic pain
Lin Yu (2015)
10.1192/bjp.bp.114.153684
Behavioural and psychological symptoms in dementia and the challenges for family carers: systematic review
Alexandra R Feast (2016)
10.1186/1471-2474-14-203
Predicting response to physiotherapy treatment for musculoskeletal shoulder pain: a systematic review
R. Chester (2013)
10.1097/PHM.0000000000001064
The Effects of Electrical Stimulation Parameters in Managing Spasticity After Spinal Cord Injury: A Systematic Review
A. H. Bekhet (2019)
10.1016/j.ijnurstu.2013.02.005
Factors associated with quality of life of people with dementia in long-term care facilities: a systematic review.
H. Beerens (2013)
10.1016/j.surg.2015.11.029
The effect of early mobilization protocols on postoperative outcomes following abdominal and thoracic surgery: A systematic review.
Tanya Castelino (2016)
10.1002/msc.1166
Treatment effectiveness and fidelity of manual therapy to the knee: A systematic review and meta‐analysis
P. Salamh (2017)
10.1371/journal.pone.0155277
Muscle Activity Onset Prior to Landing in Patients after Anterior Cruciate Ligament Injury: A Systematic Review and Meta-Analysis
D. Theisen (2016)
Cardiac Troponins Used as Diagnostic and Prognostic Tests in Patients With Kidney Disease
E. Michos (2014)
Efetividade de programas de intervenção escolar para reduzir fatores de risco à saúde em adolescentes: uma revisão sistemática Effectiveness of intervention programs in schools to reduce health risk factors in adolescents: a systematic review
Roseane de Fátima Guimarães (2015)
The links between head injury and homelessness : a qualitative study
Gemma Findlay (2016)
10.1186/s13643-015-0035-1
Red blood cell transfusion and mortality effect in aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis protocol
S. English (2015)
10.1007/s00337-016-0108-9
Adhäsive Kapsulitis der Schulter
Rahel Signer (2016)
10.1371/journal.pone.0149024
Strategies for Rapid Muscle Fatigue Reduction during FES Exercise in Individuals with Spinal Cord Injury: A Systematic Review
Morufu Olusola Ibitoye (2016)
10.1093/NUTRIT/NUW042
Influence of dietary protein and its amino acid composition on postoperative outcomes after gastric bypass surgery: a systematic review.
Merel van den Broek (2016)
10.1080/09602011.2012.712871
Depression and anxiety in parent versus spouse caregivers of adult patients with traumatic brain injury: A systematic review
Naomi Ennis (2013)
10.1016/j.rmed.2017.08.007
Utilization of non-invasive imaging tools for assessment of peripheral skeletal muscle size and composition in chronic lung disease: A systematic review.
Dmitry Rozenberg (2017)
10.1016/j.humov.2017.10.020
Recalibration in functional perceptual-motor tasks: A systematic review.
Milou Tessa Brand (2017)
10.1186/s12887-020-02122-9
The effects of serial casting on lower limb function for children with Cerebral Palsy: a systematic review with meta-analysis
Nikki Milne (2020)
10.1016/j.mayocp.2017.11.005
Potentially Modifiable Risk Factors for Long‐Term Cognitive Impairment After Critical Illness: A Systematic Review
Amra Sakusic (2018)
Encounter-based patient decision aids. Implementation experiences and effects across online and real-world contexts
Peter Scalia (2019)
10.1002/14651858.CD010183
Occupational safety and health regulation and legislation enforcement tools for preventing occupational diseases and injuries
Christina Mischke (2012)
10.7586/JKBNS.2012.14.3.221
Quality Assessment Tools and Reporting Standards in Nursing Research
K. Kim (2012)
10.1186/2046-4053-1-66
Reporting participation rates in studies of non-pharmacological interventions for patients with chronic obstructive pulmonary disease: a systematic review
R. Sohanpal (2012)
10.1016/J.GAITPOST.2011.10.216
Potential effects of an increased aerobic capacity on walking effort and walking speed in lower limb amputees
Daphne Wezenberg (2012)
10.1007/s00586-013-3144-3
Surgery in lumbar degenerative spondylolisthesis: indications, outcomes and complications. A systematic review
F. Steiger (2013)
Infection difficile Community-Associated Clostridium Meta-Analysis of Antibiotics and the Risk of
David N. Fisman (2014)
10.1016/j.jvsv.2013.10.002
Comparative effectiveness of surgical interventions aimed at treating underlying venous pathology in patients with chronic venous ulcer.
Mahmoud B. Malas (2014)
See more
Semantic Scholar Logo Some data provided by SemanticScholar