Online citations, reference lists, and bibliographies.

The Measurement Of Observer Agreement For Categorical Data.

J. Landis, G. Koch
Published 1977 · Mathematics, Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa-type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.
This paper references
10.1080/00401706.1968.10490601
Some Further Remarks Concerning “A General Approach to the Estimation of Variance Components”
G. Koch (1968)
10.2307/2528901
Analysis of categorical data by linear models.
J. Grizzle (1969)
10.2307/2529683
An analysis for compounded functions of categorical data.
R. Forthofer (1973)
10.1111/j.1467-9574.1975.tb00259.x
A review of statistical methods in the analysis of data arising from observer reliability studies (Part II)
J. Landis (1975)
10.1080/01621459.1966.10502021
A Note on the Equivalence of Two Test Criteria for Hypotheses in Categorical Data
V. P. Bhapkar (1966)
Landis, J
Mimeo Series No. (1022)
10.1037/h0031643
Measures of response agreement for qualitative data: Some generalizations and alternatives.
R. Light (1971)
10.1007/978-1-4612-9995-0_3
Measures of Association for Cross Classifications III: Approximate Sampling Theory
L. Goodman (1963)
10.1080/01621459.1966.10480876
Assessing the Accuracy of Multivariate Observations
J. Fleiss (1966)
The analysis of categorical
D W. (1971)
10.2307/2556167
Reliability of measurements for studies of cerebrovascular atherosclerosis.
R. Loewenson (1972)
10.1093/oxfordjournals.aje.a119583
Studies on multiple sclerosis in Winnipeg, Manitoba, and New Orleans, Louisiana. II. A controlled investigation of factors in the life history of the Winnipeg patients.
K. Westlund (1953)
An application of hierarchical
J. R. Landis (1977)
Measures of response
R. J. Light (1971)
10.1037/h0026256
Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit.
J. Cohen (1968)
10.2307/2529309
A general methodology for the analysis of experiments with repeated measurement of categorical data.
G. Koch (1977)
10.2307/2528934
The analysis of categorical data from mixed models
G. Koch (1971)
10.2307/2528319
On the Hypotheses of 'No Interaction' in Contingency Tables
V. P. Bhapkar (1968)
10.2307/2529786
An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers.
J. Landis (1977)
On the analysis of contingency tables with a quantitative response.
Vasant P. Bhapkar (1968)
10.1080/00401706.1959.10489861
The Measuring Process
J. Mandel (1959)
10.1037/h0028106
Large sample standard errors of kappa and weighted kappa.
J. Fleiss (1969)
10.1037/h0031619
Measuring nominal scale agreement among many raters.
J. Fleiss (1971)
Contribution to the theory of the X2 test
J Neyman (1949)
10.1080/01621459.1948.10483261
On Estimating Precision of Measuring Instruments and Product Variability
F. Grubbs (1948)
This content downloaded by the authorized user from 192.168.82.209 on Mon, 19 Nov 2012 06:33:45 AM All use subject to JSTOR Terms and Conditions
A coefficient ofagreement for nominal scales
J. Cohen (1960)
10.1177/001316447303300309
The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability
J. Fleiss (1973)
10.1177/001316446802800205
Estimating Individual Rater Reliabilities from Analysis of Treatment Effects
J. Overall (1968)
10.1016/0010-468X(76)90037-4
A computer program for the generalized chi-square analysis of categorical data using weighted least squares (GENCAT).
J. Landis (1976)
10.2307/2281294
Statistical Theory in Research
R. L. Anderson (1952)
This content downloaded by the authorized user from 192.168.82.209 on Mon
(1977)
10.1080/00401706.1973.10489010
Errors of Measurement, Precision, Accuracy and the Statistical Comparison of Measuring Instruments
F. Grubbs (1973)
10.2307/2529549
Measuring agreement between two judges on the presence or absence of a trait.
J. Fleiss (1975)
10.1090/S0002-9947-1943-0012401-3
Tests of statistical hypotheses concerning several parameters when the number of observations is large
A. Wald (1943)
10.1080/00401706.1967.10490444
A general approach to estimation of variance components
G. Koch (1965)
10.1037/e465522008-010
A new measure of agreement between rank ordered variables.
D. Cicchetti (1972)
10.1080/00401706.1968.10490539
Hypotheses Of ‘No Interaction’ In Multi-dimensional Contingency Tables
V. P. Bhapker (1968)
A general methodology for the measurement of observer agreement when the data are categorical
J. Landis (1975)



This paper is referenced by
10.1109/JSTARS.2009.2012475
An Examination of the Effects of Spatial Resolution and Image Analysis Technique on Indirect Fuel Mapping
M. Tanase (2008)
10.1016/j.math.2008.02.010
The test-retest reliability and concurrent validity of the Subjective Complaints Questionnaire for low back pain.
J. Ford (2009)
10.1177/112972980800900408
Contrast-Enhanced Magnetic Resonance Angiography Findings Prior to Hemodialysis Vascular access Creation: A Prospective Analysis
R. N. Planken (2008)
10.1016/j.dss.2008.02.009
Analyzing unstructured text data: Using latent categorization to identify intellectual communities in information systems
Kai R. Larsen (2008)
10.1111/j.1365-3156.2008.02172.x
Inconclusive results in conventional serological screening for Chagas' disease in blood banks: evaluation of cellular and humoral response.
C. R. Furucho (2008)
10.1212/WNL.48.1.119
Accuracy of the Clinical Diagnosis of Corticobasal Degeneration
I. Litvan (1997)
10.3233/BMR-160718
Intra- and inter-rater reliability of 3D passive intervertebral motion in subjects with non-specific neck pain assessed by physical therapy students: A pilot study
G. Rossettini (2016)
IRIS: English-Irish Machine Translation System
Mihael Arcan (2016)
10.1080/02640414.2016.1227466
Running quietly reduces ground reaction force and vertical loading rate and alters foot strike technique
Xuan Phan (2017)
10.1080/02687038.2015.1081139
An investigation of aphasic naming error evolution following phonomotor treatment
Irene Minkina (2016)
10.1002/ECO.1755
Alternative stable states of tidal marsh vegetation patterns and channel complexity
Kevan B. Moffett (2016)
10.1016/j.specom.2017.01.007
The role of prosody and voice quality in indirect storytelling speech: A cross-narrator perspective in four European languages
R. Montaño (2017)
10.1145/3025453.3025659
"Algorithms ruin everything": #RIPTwitter, Folk Theories, and Resistance to Algorithmic Change in Social Media
Michael A. DeVito (2017)
10.1016/j.genhosppsych.2017.01.007
Exploratory examination of the utility of demoralization as a diagnostic specifier for adjustment disorder and major depression.
D. Kissane (2017)
10.1080/09638288.2016.1207105
Cross-cultural adaptation, reliability, and validity of the Japanese version of the Cumberland ankle instability tool.
Shun Kunugi (2017)
10.3109/17518423.2015.1066461
The Spanish version of the Alberta Infant Motor Scale: Validity and reliability analysis
Erica Morales-Monforte (2017)
10.1016/j.parint.2016.10.008
Subtype distribution of Blastocystis spp. isolated from children in Eskisehir, Turkey.
Nihal Doğan (2017)
10.1080/08870446.2016.1273356
Effectiveness and content analysis of interventions to enhance medication adherence and blood pressure control in hypertension: A systematic review and meta-analysis
E. Morrissey (2017)
10.1002/TEA.21340
An exploration of teacher learning from an educative reform‐oriented science curriculum: Case studies of teacher curriculum use
Lisa M. Marco-Bujosa (2017)
10.1007/978-3-319-76430-6_1
Benchmarking Swarm Rebalancing Algorithm for Relieving Imbalanced Machine Learning Problems
Jinyan Li (2018)
10.1016/j.procs.2018.07.282
Multi-expert analysis and validation of objective vascular tortuosity measurements
L. Ramos (2018)
10.1155/2018/8491057
Tongue Image Database Construction Based on the Expert Opinions: Assessment for Individual Agreement and Methods for Expert Selection
Zhen Qi (2018)
10.1093/ejo/cjn081
Häävikko's method to assess dental age in Italian children.
A. Butti (2009)
10.1016/j.ejrad.2018.01.002
Evaluation of an automated breast volume scanner according to the fifth edition of BI-RADS for breast ultrasound compared with hand-held ultrasound.
Eun Jung Choi (2018)
10.1002/pbc.26840
Could we use parent report as a valid proxy of child report on anxiety, depression, and distress? A systematic investigation of father–mother–child triads in children successfully treated for leukemia
C. Abate (2018)
10.1259/DMFR.20170160
Evidence of genotoxicity and cytotoxicity of X-rays in the oral mucosa epithelium of adults subjected to cone beam CT.
da Fonte Jb (2018)
10.1108/ET-11-2016-0176
Internalizing the spirit of entrepreneurship in early childhood education through traditional games
Muhammad Jufri (2018)
10.1080/09593985.2017.1423428
The reliability and validity of the standardized Mensendieck test in relation to disability in patients with chronic pain
Paul Keessen (2018)
10.5194/tc-2019-190
Accuracy and Inter-Analyst Agreement of Visually Estimated Sea Ice Concentrations in Canadian Ice Service Ice Charts
Angela Cheng (2019)
10.1016/J.CITIES.2016.06.006
City dweller aspirations for cities of the future: How do environmental and personal wellbeing feature?
H. Joffe (2016)
10.1177/0007650316676270
Contextualizing Individual Competencies for Managing the Corporate Social Responsibility Adaptation Process: The Apparent Influence of the Business Case Logic
E. R. Osagie (2019)
10.1177/0146167219896136
Mindfulness and Its Association With Varied Types of Motivation: A Systematic Review and Meta-Analysis Using Self-Determination Theory
James N Donald (2019)
See more
Semantic Scholar Logo Some data provided by SemanticScholar