Deep-learning based classification distinguishes sarcomatoid malignant mesotheliomas from benign spindle cell mesothelial proliferations

Our plain-English summary of Deep-learning based classification distinguishes sarcomatoid malignant mesotheliomas from benign spindle cell mesothelial proliferations, by J.R. Naso, A.B. Levine, H. Farahani, et al (Modern Pathology, 2021).

The study

• Authors: J.R. Naso, A.B. Levine, H. Farahani, L.R. Chirieac, S. Dacic, J.L. Wright, C. Lai, H.M. Yang, S.J.M. Jones, A. Bashashati, S. Yip, A. Churg
• Journal: Modern Pathology
• Published: 2021;34(11):2028-2035
• Cited by: 18 articles, as counted by NIH iCite
• Read it: PubMed · Publisher

Abstract

Sarcomatoid mesothelioma is an aggressive malignancy that can be challenging to distinguish from benign spindle cell mesothelial proliferations based on biopsy, and this distinction is crucial to patient treatment and prognosis. A novel deep learning based classifier may be able to aid pathologists in making this critical diagnostic distinction. SpindleMesoNET was trained on cases of malignant sarcomatoid mesothelioma and benign spindle cell mesothelial proliferations. Performance was assessed through cross-validation on the training set, on an independent set of challenging cases referred for expert opinion ('referral' test set), and on an externally stained set from outside institutions ('externally stained' test set). SpindleMesoNET predicted the benign or malignant status of cases with AUC's of 0.932, 0.925, and 0.989 on the cross-validation, referral and external test sets, respectively. The accuracy of SpindleMesoNET on the referral set cases (92.5%) was comparable to the average accuracy of 3 experienced pathologists on the same slide set (91.7%). We conclude that SpindleMesoNET can accurately distinguish sarcomatoid mesothelioma from benign spindle cell mesothelial proliferations. A deep learning system of this type holds potential for future use as an ancillary test in diagnostic pathology.

© 2021. The Author(s), under exclusive licence to United States & Canadian Academy of Pathology.

In plain English

The authors built and tested SpindleMesoNET, a deep learning program that reads scanned slides to judge whether a spindle cell mesothelial growth is sarcomatoid malignant mesothelioma or benign. It was trained on 58 malignant and 81 benign cases. It was tested on 40 difficult cases sent for expert opinion, and on 25 sarcomatoid mesothelioma and 14 benign slides from two other institutions chosen as clear examples. Its AUC (1.0 is perfect) was 0.932 (95% confidence interval 0.895 to 0.967) in five-fold cross-validation on the training cases. It was 0.925 (0.836 to 0.983) for referral cases and 0.989 (0.967 to 1.0) for outside slides. On the referral cases, at the score cutoff (0.95) that maximized accuracy, it was correct for 37 of 40 (92.5%, interval 85.0 to 97.5). Three experienced pathologists, judging only the same slide, averaged 91.7%. Its three errors were all malignant cases called benign. The authors concluded it can accurately make this distinction and may serve as a supporting test. They noted it was not designed to separate sarcomatoid mesothelioma from other malignant spindle cell tumors and was built mostly on pleural cases.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Diagnostic accuracy study: development of a deep learning (neural network) classifier using retrospectively identified archived slides, with five-fold cross-validation, two separate test sets, and comparison with three pathologists
• Who was studied: Training/cross-validation set: 58 malignant mesothelioma cases (16 sarcomatoid, 15 desmoplastic, 5 mixed sarcomatoid and desmoplastic, 20 biphasic, 2 transitional; mostly pleural biopsies) and 81 benign spindle cell mesothelial proliferations, from Vancouver General Hospital archives and an expert's consultation files. Referral test set: 40 cases referred for expert opinion (19 benign, 21 malignant, all pleural). Externally stained test set: 25 pleural sarcomatoid mesotheliomas and 14 benign pleural proliferations from two other institutions, selected as clear examples of benign and malignant histology. Dates are not given.
• Exposure: Not studied. The article does not report asbestos or any other exposure. What was tested was the SpindleMesoNET computer classifier applied to H&E-stained slide images.
• Main results:
  • Five-fold cross-validation on the training set, in which patients were split into non-overlapping training and testing sets, gave an overall AUC of 0.932 (95% confidence interval 0.895 to 0.967).
  • Referral test set (40 difficult cases): AUC 0.925 (95% confidence interval 0.836 to 0.983).
  • Externally stained test set (25 sarcomatoid mesothelioma and 14 benign slides, selected as clear representations of benign and malignant processes): AUC 0.989 (95% confidence interval 0.967 to 1.0).
  • On the referral set, SpindleMesoNET accuracy was 92.5% (37/40; 95% confidence interval 85.0 to 97.5). This used the score threshold that maximized accuracy (0.95). The average accuracy of three experienced pathologists was 91.7%; their individual accuracies were 87.5%, 95.0% and 92.5%.
  • The pathologists independently predicted benign or malignant status from the histology of the same slide the program evaluated, with no other information.
  • On the referral set, SpindleMesoNET had 100% specificity and 85.7% sensitivity for malignancy. The pathologists' mean specificity was 96.5% and their mean sensitivity was 87.3%.
  • SpindleMesoNET's only errors on the referral set were three malignant cases called benign. One of these was also called incorrectly by a pathologist.
  • 9 of 40 referral cases were called incorrectly by at least one pathologist (2 benign, 7 malignant). SpindleMesoNET gave the correct diagnosis for 8 of these 9.
  • Counting a case as malignant if either SpindleMesoNET or a pathologist called it malignant gave a sensitivity of 95 to 100%, depending on the pathologist. This was higher than either alone.
  • In the training set, scores for malignant cases were not significantly associated with anatomic site, morphologic subtype, or whether the case was in-house or referred.
• The authors' conclusion: The authors concluded that SpindleMesoNET can accurately distinguish sarcomatoid mesothelioma from benign spindle cell mesothelial proliferations. They said a system of this type holds potential for future use as an ancillary (supporting) test in diagnostic pathology.
• Limitations the authors state:
  • SpindleMesoNET is not intended to distinguish sarcomatoid mesothelioma from other malignant spindle cell tumors.
  • Development used mostly pleural cases. Use on non-pleural samples would likely need further training and testing.
  • Before clinical use, the program needs training and testing on more externally stained cases from multiple institutions.
  • Annotation is labor-intensive, which restricts training set size. It limits training to areas pathologists can already recognize as malignant, and it may introduce bias from the annotating pathologists.
  • Performance may be suboptimal on samples with very low tumor content.
  • The rarity of sarcomatoid mesothelioma limits the number of samples available.

Topics

The National Library of Medicine indexes this study under subjects our site covers: Mesothelioma.

Medical terms in this study, explained in our Lexicon: Sarcomatoid, Malignant Mesothelioma, Spindle Cell, Mesothelioma, Biopsy, Prognosis, Pathology.

Related studies

• Deep learning-based classification of mesothelioma improves prediction of patient outcome — P. Courtiol, C. Maussion, et al. (2019)
• Asbestos Detection with Fluorescence Microscopy Images and Deep Learning — C. Cai, T. Nishimura, et al. (2021)
• Deep Learning for the Automatic Quantification of Pleural Plaques in Asbestos-Exposed Subjects — I. Benlala, B.D. De Senneville, et al. (2022)
• The differential diagnosis between pleural sarcomatoid mesothelioma and spindle cell/pleomorphic (sarcomatoid) carcinomas of the lung: evidence-based guidelines from the International Mesothelioma Panel and the MESOPATH National Reference Center — A.M. Marchevsky, N. LeStang, et al. (2017)
• BAP1 immunohistochemistry and p16 FISH to separate benign from malignant mesothelial proliferations — B.S. Sheffield, H.C. Hwang, et al. (2015)
• IMP3 and GLUT-1 immunohistochemistry for distinguishing benign from malignant mesothelial proliferations — A.F. Lee, A.M. Gown, et al. (2013)

Browse more research in Modern Pathology, or return to medical research.