Artificial intelligence-based diagnosis of asbestosis: analysis of a database with applicants for asbestosis state aid

Our plain-English summary of Artificial intelligence-based diagnosis of asbestosis: analysis of a database with applicants for asbestosis state aid, by K.B.W. Groot Lipman, C.J. de Gooijer, T.N. Boellaard, et al (European Radiology, 2023).

The study

• Authors: K.B.W. Groot Lipman, C.J. de Gooijer, T.N. Boellaard, F. van der Heijden, R.G.H. Beets-Tan, Z. Bodalal, S. Trebeschi, J.A. Burgers
• Journal: European Radiology
• Published: 2023;33(5):3557-3565
• Cited by: 5 articles, as counted by NIH iCite
• Read it: PubMed · Publisher · Free full text

Abstract

Objectives: In many countries, workers who developed asbestosis due to their occupation are eligible for government support. Based on the results of clinical examination, a team of pulmonologists determine the eligibility of patients to these programs. In this Dutch cohort study, we aim to demonstrate the potential role of an artificial intelligence (AI)-based system for automated, standardized, and cost-effective evaluation of applications for asbestosis patients.

Methods: A dataset of n = 523 suspected asbestosis cases/applications from across the Netherlands was retrospectively collected. Each case/application was reviewed, and based on the criteria, a panel of three pulmonologists would determine eligibility for government support. An AI system is proposed, which uses thoracic CT images as input, and predicts the assessment of the clinical panel. Alongside imaging, we evaluated the added value of lung function parameters.

Results: The proposed AI algorithm reached an AUC of 0.87 (p < 0.001) in the prediction of accepted versus rejected applications. Diffusion capacity (DLCO) also showed comparable predictive value (AUC = 0.85, p < 0.001), with little correlation between the two parameters (r-squared = 0.22, p < 0.001). The combination of the imaging AI score and DLCO achieved superior performance (AUC = 0.95, p < 0.001). Interobserver variability between pulmonologists on the panel was estimated at alpha = 0.65 (Krippendorff's alpha).

Conclusion: We developed an AI system to support the clinical decision-making process for the application to the government support for asbestosis. A multicenter prospective validation study is currently ongoing to examine the added value and reliability of this system alongside the clinic panel.

Key Points: • Artificial intelligence can detect imaging patterns of asbestosis in CT scans in a cohort of patients applying for state aid. • Combining the AI prediction with the diffusing lung function parameter reaches the highest diagnostic performance. • Specific cases with fibrosis but no asbestosis were correctly classified, suggesting robustness of the AI system, which is currently under prospective validation.

© 2022. The Author(s).

In plain English

This Dutch study tested whether an artificial intelligence (AI) system reading CT scans could predict the verdicts of the three-pulmonologist panel that judges asbestosis state aid applications. The authors retrospectively analysed 523 applications from 05/2014 to 11/2019. Of the 507 with a panel verdict, 233 (46.0%) were accepted. Panel agreement was estimated at alpha = 0.65, which the authors noted is below the cutoff considered sufficient for reliable results. After exclusions, 419 cases were used to build and test the AI. The AI separated accepted from rejected applications with an AUC of 0.87 (95% confidence interval 0.78 to 0.94). The lung function test DLCO (diffusing capacity for carbon monoxide) alone reached an AUC of 0.85 (0.80 to 0.89). The two were only weakly correlated. Averaging the AI score with one minus the DLCO value gave an AUC of 0.95 (0.89 to 0.98). The authors noted the panel's law-driven verdict does not entirely match a clinical diagnosis. They concluded that the CT-plus-DLCO model was superior to the other models; the analysis was only retrospectively validated, so they chose to test it prospectively.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Retrospective analysis of prospectively collected applications, used to develop and test an AI model (diagnostic study)
• Who was studied: 523 applications for Dutch asbestosis financial support, collected by the Dutch Institute of Asbestos Victims (IAS) and the Netherlands Cancer Institute between 05/2014 and 11/2019. Median age was 75 years and two applicants (0.4%) were female. 507 had a panel verdict. After exclusions (78 without fully imaged lungs, 10 with CT slice thickness over 5 mm), 419 formed the AI study dataset.
• Exposure: Applicants for asbestosis state aid. Dutch criteria require occupational asbestos exposure of at least five fiber years. Fiber types and individual exposure levels were not reported, and exposure was not an input to the AI. The model's inputs were chest CT scans and, in the combined models, DLCO.
• Main results:
  • Panel verdicts: of 507 applications with a verdict, 233 (46.0%) were accepted, 166 of them unanimously. The remaining 274 did not meet the criteria, 219 of them unanimously.
  • Agreement among the three pulmonologists was estimated at Krippendorff's alpha = 0.65, with 75.9% of verdicts unanimous (n = 385). The authors stated this level is lower than the cutoff considered sufficient for reliable results.
  • The best CT-only AI model (soft labels with anomaly heatmap) reached an AUC of 0.87 (CI 0.78 to 0.94) for accepted versus rejected applications.
  • This AI model had a sensitivity of 0.76 (0.62 to 0.88) and a specificity of 0.87 (0.77 to 0.96).
  • DLCO alone reached an AUC of 0.85 (CI 0.80 to 0.89). Other lung function measures scored lower: VC 0.67, FVC 0.63, and AMA class 0.83.
  • DLCO showed a weak correlation with the AI prediction (r-squared = 0.22, p less than 0.001).
  • The simple combination was defined as the average of the AI score and one minus the DLCO value.
  • The simple combination reached an AUC of 0.95 (0.89 to 0.98), with sensitivity 0.77 and specificity 0.91. An advanced combination, with DLCO added as an AI input, reached an AUC of 0.92.
  • In 5 cases all three pulmonologists were positive but the model was negative. Most of these (n = 4) had a severe reduction in lung function that the fibrosis on CT did not reflect.
• The authors' conclusion: The authors concluded that the model based on the CT scan and DLCO was superior to the other models and reached excellent diagnostic accuracy. They stated that whether the method could be used in other diagnostic settings for asbestosis or interstitial lung disease is under investigation.
• Limitations the authors state:
  • Lung function values were missing for some applicants, so each lung function result was computed on slightly different sub-cohorts.
  • Hardware limits forced downsampling of the CT images, which blurred finer structures such as fibrosis.
  • The model lacks explainability. Saliency maps show where it is looking but not why an application was accepted or rejected.
  • Cumulative asbestos exposure was not included as an input.
  • The analysis was only retrospectively validated. The authors chose to validate the AI plus DLCO combination prospectively (PROSBEST, Trial NL9064).
  • The law-driven panel verdict that the AI was trained to predict does not coincide entirely with a clinical multidisciplinary diagnosis.

Topics

The National Library of Medicine indexes this study under subjects our site covers: Asbestosis.

Medical terms in this study, explained in our Lexicon: Cohort Study, Diffusing Capacity (DLCO), CT Scan, Fibrosis.

Related studies

• Artificial Intelligence-based Quantification of Pleural Plaque Volume and Association With Lung Function in Asbestos-exposed Patients — K.B.W. Groot Lipman, T.N. Boellaard, et al. (2024)
• Development of rapid and highly accurate method to measure concentration of fibers in atmosphere using artificial intelligence and scanning electron microscopy — Y. Iida, K. Watanabe, et al. (2021)
• Rapid fiber-detection technique by artificial intelligence in phase-contrast microscope images of simulated atmospheric samples — T. Yamamoto, K. Iwasaki, et al. (2024)
• Differential diagnosis of pleural mesothelioma using Logic Learning Machine — S. Parodi, R. Filiberti, et al. (2015)
• Compensation, radiographic changes, and survival in applicants for asbestosis compensation — W.O. Cookson, A.W. Musk, et al. (1985)
• Predicting cancer using supervised machine learning: Mesothelioma — A. Choudhury (2021)

Browse more research in European Radiology, or return to medical research.