Interobserver Variability of Quantitative and Qualitative Assessment Using MRI in Malignant Pleural Mesothelioma

Our plain-English summary of Interobserver Variability of Quantitative and Qualitative Assessment Using MRI in Malignant Pleural Mesothelioma, by R.R. Gill, D.J. Murphy, R.T. Seethamraju, et al (Radiology. Cardiothoracic Imaging, 2020).

The study

• Authors: R.R. Gill, D.J. Murphy, R.T. Seethamraju, E. Mazzola, R. Bueno, W.G. Richards
• Journal: Radiology. Cardiothoracic Imaging
• Published: 2020;2(2):e190066
• Cited by: 5 articles, as counted by NIH iCite
• Read it: PubMed · Publisher · Free full text

Abstract

Purpose: To evaluate the interobserver variability associated with quantitative and qualitative MRI assessments of malignant pleural mesothelioma (MPM).

Materials And Methods: Patients with MPM who underwent uniform-protocol preoperative MRI between 2009 and 2014 were included. The MRI-derived tumor volume was estimated. Unidimensional measurements of maximal pleural thickness (P max) and average pleural thickness (P avg) on axial MR images; maximal fissural thickness (F max); maximal diaphragmatic thickness (D max); and average diaphragmatic thickness (D avg) on sagittal reconstructed images were acquired. Interobserver agreement regarding the American Joint Committee on Cancer (AJCC) tumor stage at each criterion level was assessed by using Cohen κ statistics. Agreement between quantitative measurements was assessed by using Bland-Altman plots and intraclass correlation coefficients (ICCs).

Results: The study cohort included 349 patients (median age, 68 years [age range, 30-90 years), 273 (78%) of whom were men and 203 (58%) of whom had epithelioid-subtype tumors. Qualitative assessment performed by using the AJCC staging criteria (eighth edition) was concordant in 31% of cases and yielded considerable disagreement (κ = 0.177). Inspection of the Bland-Altman plots led to decisive agreement between the two reviewers regarding MRI-derived tumor volume (ICC, 0.979). There was also a good degree of agreement between the two reviewers regarding unidimensional measurements of D max (ICC, 0.807), D avg (ICC, 0.823), P max (ICC, 0.787), P avg (ICC, 0.787), and F max (ICC, 0.659).

Conclusion: Quantitative assessment can enhance the clinical staging of MPM. Compared with qualitative assessment, quantitative assessment has low interobserver variability and could yield a tumor size criterion that is currently lacking in the AJCC clinical staging of MPM.Supplemental material is available for this article.© RSNA, 2020.

2020 by the Radiological Society of North America, Inc.

In plain English

This single-institution study measured how well two radiologists agreed reading the same preoperative MRI scans of patients with malignant pleural mesothelioma. Of 472 patients scanned with a uniform protocol from 2009 to 2014, 349 with complete data were studied; 78% were men and 70% self-reported asbestos exposure. Readers had 10 and 2 years' experience. Each independently staged every tumor (AJCC eighth edition) and measured tumor volume and pleural, fissural and diaphragmatic thickness. Their stage assignments matched in 31% of cases (kappa 0.177, 95% CI 0.119 to 0.235), which the authors called slight agreement but reflective of a high level of disagreement. The intraclass correlation coefficient was 0.979 (95% CI 0.975 to 0.983) for tumor volume. For the thickness measures it ranged from 0.659 (95% CI 0.596 to 0.715) for maximal fissural thickness to 0.823 (0.785 to 0.854) for average diaphragmatic thickness; median values differed significantly between readers for each. The authors concluded that quantitative MRI measures showed lower interobserver variability than qualitative staging and could supply a tumor size criterion lacking in AJCC staging. They noted that only two readers took part.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Retrospective, single-institution study of agreement between two radiologists (interobserver variability) reading the same MRI scans
• Who was studied: 349 patients with malignant pleural mesothelioma referred for surgical evaluation who had preoperative MRI with a uniform protocol between 2009 and 2014. They were drawn from 472 scanned patients, after 123 were excluded for incomplete data. Median age was 68 years (range 30 to 90). 273 (78%) were men and 203 (58%) had epithelioid-subtype tumors.
• Exposure: This was not a study of asbestos exposure. It assessed readings of gadolinium-enhanced 3-T MRI. Self-reported asbestos exposure was recorded for the 349 patients: yes 244 (70%), no 77 (22%), unknown 28 (8%). No fiber types were given.
• Main results:
  • The two readers' AJCC eighth-edition clinical stage assignments were concordant in 31% of the 349 cases. Cohen kappa was 0.177 (95% CI: 0.119, 0.235), which the authors described as slight agreement but reflective of a high level of disagreement.
  • MRI-derived tumor volume had an intraclass correlation coefficient of 0.979 (95% CI: 0.975, 0.983). It was the only measure without a significant difference in median values between readers (Wilcoxon P = .066).
  • Median tumor volume was 689.8 cm3 (range 15.6 to 3202.4) for reader 1 and 686.6 cm3 (range 11.8 to 3222.1) for reader 2.
  • Intraclass correlation coefficients for the thickness measurements were: maximal pleural thickness 0.787 (0.743, 0.825), average pleural thickness 0.787 (0.743, 0.825), maximal fissural thickness 0.659 (0.596, 0.715), maximal diaphragmatic thickness 0.807 (0.762, 0.843), and average diaphragmatic thickness 0.823 (0.785, 0.854). The authors wrote that these showed a more pronounced level of disagreement than tumor volume.
  • Median values differed significantly between the two readers (P less than .05) for all thickness measurements.
  • Differences in how the readers perceived involvement of the endothoracic fascia, lung parenchyma and chest wall contributed to discordant staging.
• The authors' conclusion: The authors concluded that quantitative metrics from an optimized MRI protocol showed low interobserver variability compared with qualitative staging. They wrote that such metrics could supply a tumor size criterion currently lacking in AJCC clinical staging. They added that it is not yet clear whether or how such measurements may be incorporated into staging.
• Limitations the authors state:
  • Only gadolinium-enhanced, uniform-protocol MRI was studied. Variability of MRI staging without contrast enhancement was not evaluated.
  • MRI varies across imaging units and institutions, which may limit how widely the findings apply.
  • MRI is not universally available, costs more than CT, and needs longer imaging time and specific expertise.
  • Complete histopathologic and prognostic information was not available for all patients, because the cohort received different treatments.
  • Only two readers assessed the metrics. The authors recommended a larger study.

Topics

Medical terms in this study, explained in our Lexicon: Pleural Mesothelioma, Pleural Thickness, AJCC, Epithelioid, Staging.

Related studies

• Malignant mesothelioma of the pleura: interobserver variability — A. Andrion, C. Magnani, et al. (1995)
• Interobserver variability in analysis of asbestos fibres and asbestos bodies in human lung tissue — F. Roberts, J.L. Mackinnon, et al. (2004)
• Interobserver variation in the assessment of the sarcomatoid and transitional components in biphasic mesotheliomas — S. Dacic, N. Le Stang, et al. (2020)
• Banding carcinogenic risks in developed countries: A procedural basis for qualitative assessment — B.W. Stewart (2008)
• Between- and within-reader variability in the assessment of pleural abnormality using the ILO 1980 international classification of pneumoconioses — J. Bourbeau, P. Ernst (1988)
• Variability of tumor area measurements for response assessment in malignant pleural mesothelioma — Z.E. Labby, C. Straus, et al. (2013)

Browse return to medical research.