Online citations, reference lists, and bibliographies.
← Back to Search

Investigating Various Thresholds As Immunohistochemistry Cutoffs For Observer Agreement

A. Ali, S. Bell, A. Bilsland, J. Slavin, Victoria Lynch, M. Elgoweini, M. H. Derakhshan, N. Jamieson, D. Chang, V. Brown, S. Denley, C. Orange, C. Mckay, R. Carter, K. Oien, F. Duthie
Published 2017 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Background: Clinical translation of immunohistochemistry (IHC) biomarkers requires reliable and reproducible cutoffs or thresholds for interpretation of immunostaining. Most IHC biomarker research focuses on the clinical relevance (diagnostic, prognostic, or predictive utility) of cutoffs, with less emphasis on observer agreement using these cutoffs. From the literature, we identified 3 commonly used cutoffs of 10% positive epithelial cells, 20% positive epithelial cells, and moderate to strong staining intensity (+2/+3 hereafter) to use for investigating observer agreement. Materials and Methods: A series of 36 images of microarray cores stained for 4 different IHC biomarkers, with variable staining intensity and percentage of positive cells, was used for investigating interobserver and intraobserver agreement. Seven pathologists scored the immunostaining in each image using the 3 cutoffs for positive and negative staining. Kappa (&kgr;) statistic was used to assess the strength of agreement for each cutoff. Results: The interobserver agreement between all 7 pathologists using the 3 cutoffs was reasonably good, with mean &kgr; scores of 0.64, 0.59, and 0.62, respectively, for 10%, 20%, and +2/+3 cutoffs. A good agreement was observed for experienced pathologists using the 10% cutoff, and their agreement was statistically higher than for junior pathologists (P=0.02). In addition, the mean intraobserver agreement for all 7 pathologists using the 3 cutoffs was reasonably good, with mean &kgr; scores of 0.71, 0.60, and 0.73, respectively, for 10%, 20%, and +2/+3 cutoffs. For all 3 cutoffs, a positive correlation was observed with perceived ease of interpretation (P<0.003). Finally, cytoplasmic-only staining achieved higher agreement using all 3 cutoffs than mixed staining patterns. Conclusions: All 3 cutoffs investigated achieve reasonable strength of agreement, modestly decreasing interobserver and intraobserver variability in IHC interpretation. These cutoffs have previously been used in cancer pathology, and this study provides evidence that these cutoffs can be reproducible between practicing pathologists.
This paper references
Diagnostic Utility of S100P and von Hippel-Lindau Gene Product (pVHL) in Pancreatic Adenocarcinoma—With Implication of Their Roles in Early Tumorigenesis
F. Lin (2008)
Epidermal growth factor receptor immunohistochemistry
F. Hirsch (2008)
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decision-making about cancer treatments.
M. Mazumdar (2000)
Using a higher cutoff for the percentage of HER2+ cells decreases interobserver variability in the interpretation of HER2 immunohistochemical analysis.
O. Hameed (2008)
HER-2/neu in Breast Cancer: Interobserver Variability and Performance of Immunohistochemistry with 4 Antibodies Compared with Fluorescent In Situ Hybridization
T. A. Thomson (2001)
Interobserver Variability in Histologic Recognition, Interpretation of KIT Immunostaining, and Determining MIB-1 Labeling Indices in Gastrointestinal Stromal Tumors and Other Spindle Cell Tumors of the Gastrointestinal Tract
U. Yamaguchi (2006)
Intra- and interobserver reproducibility of interpretation of immunohistochemical stains of prostate cancer
Sara Jonmarker Jaraj (2009)
S 100 P and XIAP expression in pancreatic ductal adenocarcinoma : potential novel biomarkers as a diagnostic adjunct to fine needle aspiration cytology
O Kosarac (2011)
S100P and XIAP Expression in Pancreatic Ductal Adenocarcinoma: Potential Novel Biomarkers as a Diagnostic Adjunct to Fine Needle Aspiration Cytology
Ognjen Kosarac (2011)
Expression of vascular endothelial growth factor can predict eventfree survival in stage II colon cancer
S Cascinu (2000)
The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve.
N. Perkins (2006)
Expression of KOC, S100P, mesothelin and MUC1 in pancreatico-biliary adenocarcinomas: development and utility of a potential diagnostic immunohistochemistry panel
A. Ali (2014)
Ki67 and p53 immunohistochemistry reduces interobserver variation in assessment of Barrett's oesophagus
E. Lörinc (2005)
Best immunohistochemical panel in distinguishing adenocarcinoma from squamous cell carcinoma of lung: tissue microarray assay in resected lung cancer specimens.
M. Kim (2013)
Preclinical evaluation of carcinoembryonic cell adhesion molecule (CEACAM) 6 as potential therapy target for pancreatic adenocarcinoma
Laura A Strickland (2009)
American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer.
A. Wolff (2007)
An international Ki67 reproducibility study.
Mei-Yin C. Polley (2013)
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decisionmaking about cancer
MazumdarM (2000)
Role of lymphovascular invasion and immunohistochemical expression of IMP3 in the risk stratification of superficially invasive pT1 esophageal adenocarcinoma.
A. Trivedi (2014)
Performance of p16/Ki-67 Immunostaining to Detect Cervical Cancer Precursors in a Colposcopy Referral Population
N. Wentzensen (2012)
Interobserver agreement in the reporting of colorectal polyp pathology among bowel cancer screening pathologists in Wales
J. Turner (2013)
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decision-making about cancer treatments.
M. Mazumdar (2000)
The role of immunohistochemistry in the diagnosis of flat urothelial lesions: a study using CK20, CK5/6, P53, Cd138, and Her2/Neu.
S. Jung (2014)
Effects of the change in cutoff values for human epidermal growth factor receptor 2 status by immunohistochemistry and fluorescence in situ hybridization: a study comparing conventional brightfield microscopy, image analysis-assisted microscopy, and interobserver variation.
R. Atkinson (2011)
Ki-67 expression in primary breast carcinomas and their axillary lymph node metastases: clinical implications
D. Park (2007)
Estrogen receptor status by immunohistochemistry is superior to the ligand-binding assay for predicting response to adjuvant endocrine therapy in breast cancer.
J. Harvey (1999)
Clinical trial designs for evaluating the medical utility of prognostic and predictive biomarkers in oncology.
R. Simon (2010)
CD97, CD95 and Fas‐L clearly discriminate between chronic pancreatitis and pancreatic ductal adenocarcinoma in perioperative evaluation of cryocut sections
C. Boltze (2002)
Grading of dysplasia in Barrett's oesophagus: substantial interobserver variation between general and gastrointestinal pathologists
M. Kerkhof (2007)
The measurement of observer agreement for categorical data.
J. Landis (1977)
HER-2/neu gene amplification compared with HER-2/neu protein overexpression and interobserver reproducibility in invasive breast carcinoma.
M. Hoang (2000)
Validity and reproducibility of Ki‐67 assessment in gastrointestinal stromal tumors and leiomyosarcomas
J. Ogino (2013)
Consistency and inconsistency in testing biomarkers in breast cancer. A GRELL study in cut-off variability in the Romance language countries.
E. Crocetti (2013)
What Levels of Agreement Can Be Expected Between Histopathologists Assigning Cases to Discrete Nominal Categories? A Study of the Diagnosis of Hyperplastic and Adenomatous Colorectal Polyps
Simon S. Cross (2000)
Prognostic immunophenotypic biomarker studies in diffuse large B cell lymphoma with special emphasis on rational determination of cut-off scores
A. Tzankov (2010)
Immunostaining for human herpesvirus 8 latent nuclear antigen-1 helps distinguish Kaposi sarcoma from its mimickers.
W. Cheuk (2004)
Expression of vascular endothelial growth factor can predict event-free survival in stage II colon cancer.
S. Cascinu (2000)
A simple immunohistochemical panel comprising 2 conventional markers, Ki67 and p53, is a powerful tool for predicting patient outcome in luminal-type breast cancer
Takayuki Kobayashi (2013)
Receiver Operating Characteristic (ROC) to Determine Cut-Off Points of Biomarkers in Lung Cancer Patients
H. Weiss (2003)
Subcellular mislocalization of the transcription factor NF-E2 in erythroid cells discriminates prefibrotic primary myelofibrosis from essential thrombocythemia.
K. Aumann (2013)
Good interobserver and intraobserver agreement in the evaluation of the new ILAE classification of focal cortical dysplasias
R. Coras (2012)
Using Biomarkers as Objective Standards in the Diagnosis of Cervical Biopsies
Mary T Galgano (2010)
Tissue array technology for testing interlaboratory and interobserver reproducibility of immunohistochemical estrogen receptor analysis in a large multicenter trial.
R. von Wasielewski (2002)
Expression of Maspin in Pancreatic Neoplasms: Application of Maspin Immunohistochemistry to the Differential Diagnosis
Y. Oh (2002)
Selecting immunohistochemical cut-off scores for novel biomarkers of progression and survival in colorectal cancer
I. Zlobec (2006)
The predictive value of apoptosis protease‐activating factor 1 in rectal tumors treated with preoperative, high‐dose‐rate brachytherapy
I. Zlobec (2006)
Pancreatic duct cell carcinomas express high levels of high mobility group I(Y) proteins.
N. Abe (2000)
Cytokeratin 7 and Cytokeratin 20 Expression in Epithelial Neoplasms: A Survey of 435 Cases
P. Chu (2000)
Malignant mesothelioma of the pleura. The reproducibility of the immunohistological diagnosis.
P. Betta (1997)
Claudin‐4 immunohistochemistry is highly effective in distinguishing adenocarcinoma from malignant mesothelioma in effusion cytology
V. Jo (2014)
Evaluation of intra‐ and interobserver agreement and its clinical significance for scoring bcl‐2 immunohistochemical expression in diffuse large B‐cell lymphoma
Vinicius Freitas Borlot (2008)
Prognostic value of Ki-67 for prostate cancer death in a conservatively managed cohort
G. Fisher (2013)
Expression of the tumor suppressor gene Maspin in human pancreatic cancers.
N. Maass (2001)
Cutoff Finder: A Comprehensive and Straightforward Web Application Enabling Rapid Biomarker Cutoff Optimization
J. Budczies (2012)
Observer variation in immunohistochemical analysis of protein expression, time for a change?
T. Kirkegaard (2006)
Study Designs and Statistical Analyses for Biomarker Research
Masahiko Gosho (2012)
Interobserver agreement among pathologists for semiquantitative hormone receptor scoring in breast carcinoma.
D. A. Cohen (2012)
Immunohistochemical Expression of CK20, p53, and Ki-67 as Objective Markers of Urothelial Dysplasia
C. Mallofré (2003)
Immunohistochemistry in the differential diagnostics of primary lung cancer: an investigation within the Southern Swedish Lung Cancer Study.
H. Brunnström (2013)

This paper is referenced by
Semantic Scholar Logo Some data provided by SemanticScholar