Observer variability in mesothelioma tumor thickness measurements

Defining minimally measurable lesions

Samuel G. Armato*, Anna K. Nowak, Roslyn J. Francis, Masha Kocherginsky, Michael J. Byrne

*Corresponding author for this work

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition.

Original languageEnglish (US)
Pages (from-to)1187-1194
Number of pages8
JournalJournal of Thoracic Oncology
Volume9
Issue number8
DOIs
StatePublished - Jan 1 2014

Fingerprint

Mesothelioma
Neoplasms
Tomography
Clinical Trials
Observer Variation
Clinical Protocols
Analysis of Variance
Thorax
Databases
Guidelines
Confidence Intervals
Technology

Keywords

  • Interobserver variability.
  • Malignant pleural mesothelioma
  • Response assessment
  • Staging
  • Thoracic computed tomography

ASJC Scopus subject areas

  • Oncology
  • Pulmonary and Respiratory Medicine

Cite this

Armato, Samuel G. ; Nowak, Anna K. ; Francis, Roslyn J. ; Kocherginsky, Masha ; Byrne, Michael J. / Observer variability in mesothelioma tumor thickness measurements : Defining minimally measurable lesions. In: Journal of Thoracic Oncology. 2014 ; Vol. 9, No. 8. pp. 1187-1194.
@article{949a8c20e5fc46caa0481a293f8f8589,
title = "Observer variability in mesothelioma tumor thickness measurements: Defining minimally measurable lesions",
abstract = "Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1{\%} of the mean per-site measurement (SD = 9.1{\%}; median range, 13.1{\%}). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20{\%} of the tumor thickness. The 95{\%} confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in {"}minimally measurable disease{"} from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition.",
keywords = "Interobserver variability., Malignant pleural mesothelioma, Response assessment, Staging, Thoracic computed tomography",
author = "Armato, {Samuel G.} and Nowak, {Anna K.} and Francis, {Roslyn J.} and Masha Kocherginsky and Byrne, {Michael J.}",
year = "2014",
month = "1",
day = "1",
doi = "10.1097/JTO.0000000000000211",
language = "English (US)",
volume = "9",
pages = "1187--1194",
journal = "Journal of Thoracic Oncology",
issn = "1556-0864",
publisher = "International Association for the Study of Lung Cancer",
number = "8",

}

Observer variability in mesothelioma tumor thickness measurements : Defining minimally measurable lesions. / Armato, Samuel G.; Nowak, Anna K.; Francis, Roslyn J.; Kocherginsky, Masha; Byrne, Michael J.

In: Journal of Thoracic Oncology, Vol. 9, No. 8, 01.01.2014, p. 1187-1194.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Observer variability in mesothelioma tumor thickness measurements

T2 - Defining minimally measurable lesions

AU - Armato, Samuel G.

AU - Nowak, Anna K.

AU - Francis, Roslyn J.

AU - Kocherginsky, Masha

AU - Byrne, Michael J.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition.

AB - Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition.

KW - Interobserver variability.

KW - Malignant pleural mesothelioma

KW - Response assessment

KW - Staging

KW - Thoracic computed tomography

UR - http://www.scopus.com/inward/record.url?scp=84905870439&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905870439&partnerID=8YFLogxK

U2 - 10.1097/JTO.0000000000000211

DO - 10.1097/JTO.0000000000000211

M3 - Article

VL - 9

SP - 1187

EP - 1194

JO - Journal of Thoracic Oncology

JF - Journal of Thoracic Oncology

SN - 1556-0864

IS - 8

ER -