Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions: A Validation Study

Carrie Marshall, Rawad Mounzer, Matt Hall, Violette Simon, Barbara Centeno, Katie Dennis, Jasreman Dhillon, Fang Fan, Laila Khazai, Jason Klapman, Srinadh Komanduri, Xiaoqi Lin, David Lu, Sanjana Mehrotra, V. Raman Muthusamy, Ritu Nayar, Ajit Paintal, Jianyu Rao, Sharon Sams, Janak ShahRabindra Watson, Amit Rastogi, Sachin Wani*

*Corresponding author for this work

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Background & Aims: Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement. Methods: We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement. Results: The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95% CI, 0.43–0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95% CI, –0.03 to –0.04) and qualitative parameters (κ = 0.5; 95% CI, 0.47–0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95% CI, 1.1–26.89). Conclusions: There is a suboptimal level of agreement among cytopathologists in the diagnosis of malignancy based on analysis of EUS-FNA specimens obtained from solid pancreatic masses. Strategies are needed to refine the cytologic criteria for diagnosis of malignancy and enhance tissue acquisition techniques to improve diagnostic reproducibility among cytopathologists.

Original languageEnglish (US)
Pages (from-to)1114-1122.e2
JournalClinical Gastroenterology and Hepatology
Volume16
Issue number7
DOIs
StatePublished - Jul 2018

Fingerprint

Validation Studies
Endoscopic Ultrasound-Guided Fine Needle Aspiration
Needles
Neoplasms
Logistic Models
Jaundice
Terminology
Tertiary Care Centers
Multivariate Analysis
Odds Ratio
Prospective Studies

Keywords

  • Cytopathology
  • EUS-FNA
  • Interobserver Agreement
  • Pancreas

ASJC Scopus subject areas

  • Hepatology
  • Gastroenterology

Cite this

Marshall, Carrie ; Mounzer, Rawad ; Hall, Matt ; Simon, Violette ; Centeno, Barbara ; Dennis, Katie ; Dhillon, Jasreman ; Fan, Fang ; Khazai, Laila ; Klapman, Jason ; Komanduri, Srinadh ; Lin, Xiaoqi ; Lu, David ; Mehrotra, Sanjana ; Muthusamy, V. Raman ; Nayar, Ritu ; Paintal, Ajit ; Rao, Jianyu ; Sams, Sharon ; Shah, Janak ; Watson, Rabindra ; Rastogi, Amit ; Wani, Sachin. / Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions : A Validation Study. In: Clinical Gastroenterology and Hepatology. 2018 ; Vol. 16, No. 7. pp. 1114-1122.e2.
@article{df808ee4472b4a0aae916c556ddb8b79,
title = "Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions: A Validation Study",
abstract = "Background & Aims: Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement. Methods: We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement. Results: The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95{\%} CI, 0.43–0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95{\%} CI, –0.03 to –0.04) and qualitative parameters (κ = 0.5; 95{\%} CI, 0.47–0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95{\%} CI, 1.1–26.89). Conclusions: There is a suboptimal level of agreement among cytopathologists in the diagnosis of malignancy based on analysis of EUS-FNA specimens obtained from solid pancreatic masses. Strategies are needed to refine the cytologic criteria for diagnosis of malignancy and enhance tissue acquisition techniques to improve diagnostic reproducibility among cytopathologists.",
keywords = "Cytopathology, EUS-FNA, Interobserver Agreement, Pancreas",
author = "Carrie Marshall and Rawad Mounzer and Matt Hall and Violette Simon and Barbara Centeno and Katie Dennis and Jasreman Dhillon and Fang Fan and Laila Khazai and Jason Klapman and Srinadh Komanduri and Xiaoqi Lin and David Lu and Sanjana Mehrotra and Muthusamy, {V. Raman} and Ritu Nayar and Ajit Paintal and Jianyu Rao and Sharon Sams and Janak Shah and Rabindra Watson and Amit Rastogi and Sachin Wani",
year = "2018",
month = "7",
doi = "10.1016/j.cgh.2017.09.013",
language = "English (US)",
volume = "16",
pages = "1114--1122.e2",
journal = "Clinical Gastroenterology and Hepatology",
issn = "1542-3565",
publisher = "W.B. Saunders Ltd",
number = "7",

}

Marshall, C, Mounzer, R, Hall, M, Simon, V, Centeno, B, Dennis, K, Dhillon, J, Fan, F, Khazai, L, Klapman, J, Komanduri, S, Lin, X, Lu, D, Mehrotra, S, Muthusamy, VR, Nayar, R, Paintal, A, Rao, J, Sams, S, Shah, J, Watson, R, Rastogi, A & Wani, S 2018, 'Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions: A Validation Study', Clinical Gastroenterology and Hepatology, vol. 16, no. 7, pp. 1114-1122.e2. https://doi.org/10.1016/j.cgh.2017.09.013

Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions : A Validation Study. / Marshall, Carrie; Mounzer, Rawad; Hall, Matt; Simon, Violette; Centeno, Barbara; Dennis, Katie; Dhillon, Jasreman; Fan, Fang; Khazai, Laila; Klapman, Jason; Komanduri, Srinadh; Lin, Xiaoqi; Lu, David; Mehrotra, Sanjana; Muthusamy, V. Raman; Nayar, Ritu; Paintal, Ajit; Rao, Jianyu; Sams, Sharon; Shah, Janak; Watson, Rabindra; Rastogi, Amit; Wani, Sachin.

In: Clinical Gastroenterology and Hepatology, Vol. 16, No. 7, 07.2018, p. 1114-1122.e2.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions

T2 - A Validation Study

AU - Marshall, Carrie

AU - Mounzer, Rawad

AU - Hall, Matt

AU - Simon, Violette

AU - Centeno, Barbara

AU - Dennis, Katie

AU - Dhillon, Jasreman

AU - Fan, Fang

AU - Khazai, Laila

AU - Klapman, Jason

AU - Komanduri, Srinadh

AU - Lin, Xiaoqi

AU - Lu, David

AU - Mehrotra, Sanjana

AU - Muthusamy, V. Raman

AU - Nayar, Ritu

AU - Paintal, Ajit

AU - Rao, Jianyu

AU - Sams, Sharon

AU - Shah, Janak

AU - Watson, Rabindra

AU - Rastogi, Amit

AU - Wani, Sachin

PY - 2018/7

Y1 - 2018/7

N2 - Background & Aims: Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement. Methods: We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement. Results: The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95% CI, 0.43–0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95% CI, –0.03 to –0.04) and qualitative parameters (κ = 0.5; 95% CI, 0.47–0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95% CI, 1.1–26.89). Conclusions: There is a suboptimal level of agreement among cytopathologists in the diagnosis of malignancy based on analysis of EUS-FNA specimens obtained from solid pancreatic masses. Strategies are needed to refine the cytologic criteria for diagnosis of malignancy and enhance tissue acquisition techniques to improve diagnostic reproducibility among cytopathologists.

AB - Background & Aims: Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement. Methods: We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement. Results: The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95% CI, 0.43–0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95% CI, –0.03 to –0.04) and qualitative parameters (κ = 0.5; 95% CI, 0.47–0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95% CI, 1.1–26.89). Conclusions: There is a suboptimal level of agreement among cytopathologists in the diagnosis of malignancy based on analysis of EUS-FNA specimens obtained from solid pancreatic masses. Strategies are needed to refine the cytologic criteria for diagnosis of malignancy and enhance tissue acquisition techniques to improve diagnostic reproducibility among cytopathologists.

KW - Cytopathology

KW - EUS-FNA

KW - Interobserver Agreement

KW - Pancreas

UR - http://www.scopus.com/inward/record.url?scp=85044760749&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044760749&partnerID=8YFLogxK

U2 - 10.1016/j.cgh.2017.09.013

DO - 10.1016/j.cgh.2017.09.013

M3 - Article

C2 - 28911946

AN - SCOPUS:85044760749

VL - 16

SP - 1114-1122.e2

JO - Clinical Gastroenterology and Hepatology

JF - Clinical Gastroenterology and Hepatology

SN - 1542-3565

IS - 7

ER -