Online citations, reference lists, and bibliographies.
← Back to Search

Investigating Various Thresholds As Immunohistochemistry Cutoffs For Observer Agreement

A. Ali, S. Bell, A. Bilsland, J. Slavin, Victoria Lynch, M. Elgoweini, M. H. Derakhshan, N. Jamieson, D. Chang, V. Brown, S. Denley, C. Orange, C. Mckay, R. Carter, K. Oien, F. Duthie
Published 2017 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
Background: Clinical translation of immunohistochemistry (IHC) biomarkers requires reliable and reproducible cutoffs or thresholds for interpretation of immunostaining. Most IHC biomarker research focuses on the clinical relevance (diagnostic, prognostic, or predictive utility) of cutoffs, with less emphasis on observer agreement using these cutoffs. From the literature, we identified 3 commonly used cutoffs of 10% positive epithelial cells, 20% positive epithelial cells, and moderate to strong staining intensity (+2/+3 hereafter) to use for investigating observer agreement. Materials and Methods: A series of 36 images of microarray cores stained for 4 different IHC biomarkers, with variable staining intensity and percentage of positive cells, was used for investigating interobserver and intraobserver agreement. Seven pathologists scored the immunostaining in each image using the 3 cutoffs for positive and negative staining. Kappa (&kgr;) statistic was used to assess the strength of agreement for each cutoff. Results: The interobserver agreement between all 7 pathologists using the 3 cutoffs was reasonably good, with mean &kgr; scores of 0.64, 0.59, and 0.62, respectively, for 10%, 20%, and +2/+3 cutoffs. A good agreement was observed for experienced pathologists using the 10% cutoff, and their agreement was statistically higher than for junior pathologists (P=0.02). In addition, the mean intraobserver agreement for all 7 pathologists using the 3 cutoffs was reasonably good, with mean &kgr; scores of 0.71, 0.60, and 0.73, respectively, for 10%, 20%, and +2/+3 cutoffs. For all 3 cutoffs, a positive correlation was observed with perceived ease of interpretation (P<0.003). Finally, cytoplasmic-only staining achieved higher agreement using all 3 cutoffs than mixed staining patterns. Conclusions: All 3 cutoffs investigated achieve reasonable strength of agreement, modestly decreasing interobserver and intraobserver variability in IHC interpretation. These cutoffs have previously been used in cancer pathology, and this study provides evidence that these cutoffs can be reproducible between practicing pathologists.
This paper references
10.1097/PAS.0b013e31815701d1
Diagnostic Utility of S100P and von Hippel-Lindau Gene Product (pVHL) in Pancreatic Adenocarcinoma—With Implication of Their Roles in Early Tumorigenesis
F. Lin (2008)
10.1002/cncr.23282
Epidermal growth factor receptor immunohistochemistry
F. Hirsch (2008)
10.1002/(SICI)1097-0258(20000115)19:1<113::AID-SIM245>3.0.CO;2-O
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decision-making about cancer treatments.
M. Mazumdar (2000)
10.1309/KETM3HANLUQHW7YV
Using a higher cutoff for the percentage of HER2+ cells decreases interobserver variability in the interpretation of HER2 immunohistochemical analysis.
O. Hameed (2008)
10.1038/modpathol.3880440
HER-2/neu in Breast Cancer: Interobserver Variability and Performance of Immunohistochemistry with 4 Antibodies Compared with Fluorescent In Situ Hybridization
T. A. Thomson (2001)
10.1097/01.PAI.0000151023.88969.D7
Interobserver Variability in Histologic Recognition, Interpretation of KIT Immunostaining, and Determining MIB-1 Labeling Indices in Gastrointestinal Stromal Tumors and Other Spindle Cell Tumors of the Gastrointestinal Tract
U. Yamaguchi (2006)
10.1007/s00428-009-0833-8
Intra- and interobserver reproducibility of interpretation of immunohistochemical stains of prostate cancer
Sara Jonmarker Jaraj (2009)
S 100 P and XIAP expression in pancreatic ductal adenocarcinoma : potential novel biomarkers as a diagnostic adjunct to fine needle aspiration cytology
O Kosarac (2011)
10.1159/000320913
S100P and XIAP Expression in Pancreatic Ductal Adenocarcinoma: Potential Novel Biomarkers as a Diagnostic Adjunct to Fine Needle Aspiration Cytology
Ognjen Kosarac (2011)
Expression of vascular endothelial growth factor can predict eventfree survival in stage II colon cancer
S Cascinu (2000)
10.1093/AJE/KWJ063
The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve.
N. Perkins (2006)
10.1186/1472-6890-14-35
Expression of KOC, S100P, mesothelin and MUC1 in pancreatico-biliary adenocarcinomas: development and utility of a potential diagnostic immunohistochemistry panel
A. Ali (2014)
10.1111/j.1365-2559.2005.02139.x
Ki67 and p53 immunohistochemistry reduces interobserver variation in assessment of Barrett's oesophagus
E. Lörinc (2005)
10.1016/j.anndiagpath.2012.07.006
Best immunohistochemical panel in distinguishing adenocarcinoma from squamous cell carcinoma of lung: tissue microarray assay in resected lung cancer specimens.
M. Kim (2013)
10.1002/path.2545
Preclinical evaluation of carcinoembryonic cell adhesion molecule (CEACAM) 6 as potential therapy target for pancreatic adenocarcinoma
Laura A Strickland (2009)
10.1043/1543-2165(2007)131[18:ASOCCO]2.0.CO;2
American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer.
A. Wolff (2007)
10.1093/jnci/djt306
An international Ki67 reproducibility study.
Mei-Yin C. Polley (2013)
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decisionmaking about cancer
MazumdarM (2000)
10.1016/j.prp.2014.01.018
Role of lymphovascular invasion and immunohistochemical expression of IMP3 in the risk stratification of superficially invasive pT1 esophageal adenocarcinoma.
A. Trivedi (2014)
10.1158/1078-0432.CCR-12-0270
Performance of p16/Ki-67 Immunostaining to Detect Cervical Cancer Precursors in a Colposcopy Referral Population
N. Wentzensen (2012)
10.1111/his.12110
Interobserver agreement in the reporting of colorectal polyp pathology among bowel cancer screening pathologists in Wales
J. Turner (2013)
10.1002/0470023678.CH2A
Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decision-making about cancer treatments.
M. Mazumdar (2000)
10.1016/j.anndiagpath.2013.10.006
The role of immunohistochemistry in the diagnosis of flat urothelial lesions: a study using CK20, CK5/6, P53, Cd138, and Her2/Neu.
S. Jung (2014)
10.5858/2010-0462-OAR
Effects of the change in cutoff values for human epidermal growth factor receptor 2 status by immunohistochemistry and fluorescence in situ hybridization: a study comparing conventional brightfield microscopy, image analysis-assisted microscopy, and interobserver variation.
R. Atkinson (2011)
10.1007/s00428-007-0435-2
Ki-67 expression in primary breast carcinomas and their axillary lymph node metastases: clinical implications
D. Park (2007)
10.1200/JCO.1999.17.5.1474
Estrogen receptor status by immunohistochemistry is superior to the ligand-binding assay for predicting response to adjuvant endocrine therapy in breast cancer.
J. Harvey (1999)
10.2217/PME.09.49
Clinical trial designs for evaluating the medical utility of prognostic and predictive biomarkers in oncology.
R. Simon (2010)
10.1046/j.1440-1827.2002.01324.x
CD97, CD95 and Fas‐L clearly discriminate between chronic pancreatitis and pancreatic ductal adenocarcinoma in perioperative evaluation of cryocut sections
C. Boltze (2002)
10.1111/j.1365-2559.2007.02706.x
Grading of dysplasia in Barrett's oesophagus: substantial interobserver variation between general and gastrointestinal pathologists
M. Kerkhof (2007)
10.2307/2529310
The measurement of observer agreement for categorical data.
J. Landis (1977)
10.1309/VACP-VLQA-G9DX-VUDF
HER-2/neu gene amplification compared with HER-2/neu protein overexpression and interobserver reproducibility in invasive breast carcinoma.
M. Hoang (2000)
10.1111/pin.12038
Validity and reproducibility of Ki‐67 assessment in gastrointestinal stromal tumors and leiomyosarcomas
J. Ogino (2013)
10.1016/j.breast.2013.04.012
Consistency and inconsistency in testing biomarkers in breast cancer. A GRELL study in cut-off variability in the Romance language countries.
E. Crocetti (2013)
10.1038/modpathol.3880171
What Levels of Agreement Can Be Expected Between Histopathologists Assigning Cases to Discrete Nominal Categories? A Study of the Diagnosis of Hyperplastic and Adenomatous Colorectal Polyps
Simon S. Cross (2000)
10.3109/10428190903370338
Prognostic immunophenotypic biomarker studies in diffuse large B cell lymphoma with special emphasis on rational determination of cut-off scores
A. Tzankov (2010)
10.1309/B8TC0LBVH8XY5MFV
Immunostaining for human herpesvirus 8 latent nuclear antigen-1 helps distinguish Kaposi sarcoma from its mimickers.
W. Cheuk (2004)
Expression of vascular endothelial growth factor can predict event-free survival in stage II colon cancer.
S. Cascinu (2000)
10.1186/1472-6890-13-5
A simple immunohistochemical panel comprising 2 conventional markers, Ki67 and p53, is a powerful tool for predicting patient outcome in luminal-type breast cancer
Takayuki Kobayashi (2013)
10.1155/2004/856026
Receiver Operating Characteristic (ROC) to Determine Cut-Off Points of Biomarkers in Lung Cancer Patients
H. Weiss (2003)
10.1182/blood-2012-11-463257
Subcellular mislocalization of the transcription factor NF-E2 in erythroid cells discriminates prefibrotic primary myelofibrosis from essential thrombocythemia.
K. Aumann (2013)
10.1111/j.1528-1167.2012.03508.x
Good interobserver and intraobserver agreement in the evaluation of the new ILAE classification of focal cortical dysplasias
R. Coras (2012)
10.1097/PAS.0b013e3181e8b2c4
Using Biomarkers as Objective Standards in the Diagnosis of Cervical Biopsies
Mary T Galgano (2010)
10.1309/URLK-6AVK-331U-0V5P
Tissue array technology for testing interlaboratory and interobserver reproducibility of immunohistochemical estrogen receptor analysis in a large multicenter trial.
R. von Wasielewski (2002)
10.1097/00129039-200203000-00011
Expression of Maspin in Pancreatic Neoplasms: Application of Maspin Immunohistochemistry to the Differential Diagnosis
Y. Oh (2002)
10.1136/jcp.2006.044537
Selecting immunohistochemical cut-off scores for novel biomarkers of progression and survival in colorectal cancer
I. Zlobec (2006)
10.1002/cncr.21600
The predictive value of apoptosis protease‐activating factor 1 in rectal tumors treated with preoperative, high‐dose‐rate brachytherapy
I. Zlobec (2006)
Pancreatic duct cell carcinomas express high levels of high mobility group I(Y) proteins.
N. Abe (2000)
10.1038/modpathol.3880175
Cytokeratin 7 and Cytokeratin 20 Expression in Epithelial Neoplasms: A Survey of 435 Cases
P. Chu (2000)
10.1016/S0344-0338(97)80054-4
Malignant mesothelioma of the pleura. The reproducibility of the immunohistological diagnosis.
P. Betta (1997)
10.1002/cncy.21392
Claudin‐4 immunohistochemistry is highly effective in distinguishing adenocarcinoma from malignant mesothelioma in effusion cytology
V. Jo (2014)
10.1111/j.1440-1827.2008.02276.x
Evaluation of intra‐ and interobserver agreement and its clinical significance for scoring bcl‐2 immunohistochemical expression in diffuse large B‐cell lymphoma
Vinicius Freitas Borlot (2008)
10.1038/bjc.2012.598
Prognostic value of Ki-67 for prostate cancer death in a conservatively managed cohort
G. Fisher (2013)
Expression of the tumor suppressor gene Maspin in human pancreatic cancers.
N. Maass (2001)
10.1371/journal.pone.0051862
Cutoff Finder: A Comprehensive and Straightforward Web Application Enabling Rapid Biomarker Cutoff Optimization
J. Budczies (2012)
10.1111/j.1365-2559.2006.02412.x
Observer variation in immunohistochemical analysis of protein expression, time for a change?
T. Kirkegaard (2006)
10.3390/s120708966
Study Designs and Statistical Analyses for Biomarker Research
Masahiko Gosho (2012)
10.1309/AJCP6DKRND5CKVDD
Interobserver agreement among pathologists for semiquantitative hormone receptor scoring in breast carcinoma.
D. A. Cohen (2012)
10.1097/01.MP.0000056628.38714.5D
Immunohistochemical Expression of CK20, p53, and Ki-67 as Objective Markers of Urothelial Dysplasia
C. Mallofré (2003)
10.1309/AJCP50RDXSCSBTBO
Immunohistochemistry in the differential diagnostics of primary lung cancer: an investigation within the Southern Swedish Lung Cancer Study.
H. Brunnström (2013)



This paper is referenced by
Semantic Scholar Logo Some data provided by SemanticScholar