Predictive accuracy of the Liverpool Lung Project risk model for stratifying patients for computed tomography screening for lung cancer: a case-control and cohort validation study

Our plain-English summary of Predictive accuracy of the Liverpool Lung Project risk model for stratifying patients for computed tomography screening for lung cancer: a case-control and cohort validation study, by O.Y. Raji, S.W. Duffy, O.F. Agbaje, et al (Annals of Internal Medicine, 2012).

The study

• Authors: O.Y. Raji, S.W. Duffy, O.F. Agbaje, S.G. Baker, D.C. Christiani, A. Cassidy, J.K. Field
• Journal: Annals of Internal Medicine
• Published: 2012;157(4):242-50
• Cited by: 151 articles, as counted by NIH iCite
• Read it: PubMed · Publisher · Free full text

Abstract

Background: External validation of existing lung cancer risk prediction models is limited. Using such models in clinical practice to guide the referral of patients for computed tomography (CT) screening for lung cancer depends on external validation and evidence of predicted clinical benefit.

Objective: To evaluate the discrimination of the Liverpool Lung Project (LLP) risk model and demonstrate its predicted benefit for stratifying patients for CT screening by using data from 3 independent studies from Europe and North America.

Design: Case-control and prospective cohort study.

Setting: Europe and North America.

Patients: Participants in the European Early Lung Cancer (EUELC) and Harvard case-control studies and the LLP population-based prospective cohort (LLPC) study.

Measurements: 5-year absolute risks for lung cancer predicted by the LLP model.

Results: The LLP risk model had good discrimination in both the Harvard (area under the receiver-operating characteristic curve [AUC], 0.76 [95% CI, 0.75 to 0.78]) and the LLPC (AUC, 0.82 [CI, 0.80 to 0.85]) studies and modest discrimination in the EUELC (AUC, 0.67 [CI, 0.64 to 0.69]) study. The decision utility analysis, which incorporates the harms and benefit of using a risk model to make clinical decisions, indicates that the LLP risk model performed better than smoking duration or family history alone in stratifying high-risk patients for lung cancer CT screening.

Limitations: The model cannot assess whether including other risk factors, such as lung function or genetic markers, would improve accuracy. Lack of information on asbestos exposure in the LLPC limited the ability to validate the complete LLP risk model.

Conclusion: Validation of the LLP risk model in 3 independent external data sets demonstrated good discrimination and evidence of predicted benefits for stratifying patients for lung cancer CT screening. Further studies are needed to prospectively evaluate model performance and evaluate the optimal population risk thresholds for initiating lung cancer screening.

In plain English

This study tested the Liverpool Lung Project (LLP) 5-year lung cancer risk model in three independent data sets: the European Early Lung Cancer (EUELC) case–control study (8 countries), the Harvard case–control study (Boston) and a Liverpool cohort of 7652 people. It uses smoking duration, pneumonia history, other cancer history, family history of lung cancer and asbestos exposure. Asbestos information was missing for most cohort members, so a version without asbestos was used. The model's ability to separate people who did and did not develop lung cancer (AUC, where 0.5 means chance and 1 means perfect) was 0.67 (95% CI 0.64 to 0.69) in EUELC, 0.76 (0.75 to 0.78) in Harvard and 0.82 (0.80 to 0.85) in the cohort, higher than smoking duration alone. Its predicted net benefit exceeded the other strategies tested at risk thresholds from 3% to 15%. Asbestos exposure was reported by 20.0% of EUELC case patients versus 5.8% of controls, and 23.2% versus 1.1% at Harvard. The authors concluded the model may help select high-risk patients for CT screening but needs further prospective evaluation.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: External validation of an existing lung cancer risk prediction model in 2 case–control studies and 1 population-based prospective cohort study, with decision curve and relative utility curve analysis
• Who was studied: EUELC case–control study: 585 case patients with surgically resected non–small cell lung cancer and their controls, in 8 European countries, 2002 to 2006. Harvard case–control study: 2922 participants (1738 lung cancer patients, 1184 controls) at Massachusetts General Hospital since 1992. LLPC cohort: 7652 people aged 40 to 79 years from the Liverpool area or local hospitals, recruited 1998 to 2008.
• Exposure: Self-reported risk factors entered into the LLP model: smoking duration, history of pneumonia, history of cancer, family history of lung cancer, and occupational asbestos exposure. No fiber types are named. Asbestos exposure was missing for 97% of LLPC participants, so the cohort was analysed with a model that left asbestos out.
• Main results:
  • The model's discrimination (AUC, 95% CI) was 0.67 (0.64–0.69) in EUELC, 0.76 (0.75–0.78) in Harvard and 0.82 (0.80–0.85) in the LLPC cohort.
  • Smoking duration alone had an AUC of 0.63 in EUELC, 0.74 in Harvard and 0.72 in LLPC.
  • In the LLPC cohort, 420 of 7652 participants developed lung cancer over an average follow-up of 8 years.
  • Across all three data sets, the model had greater predicted net benefit than every alternative strategy tested (screening everyone, screening no one, smoking duration alone, family history alone) at absolute risk thresholds from 3% to 15%.
  • At a 5% risk threshold, the model gave a higher proportion of true-positive classifications than screening everyone: 2.3% higher in LLPC and 3% higher in EUELC.
  • Reported asbestos exposure in the validation case–control studies: EUELC, 117 case patients (20.0%) versus 74 controls (5.8%); Harvard, 403 case patients (23.2%) versus 13 controls (1.1%).
  • The model had moderate overall calibration, with better accuracy at higher predicted risks.
• The authors' conclusion: The authors concluded that the LLP model is simple, uses few variables and may be useful for selecting high-risk patients for lung cancer CT screening. They also concluded that further prospective evaluation is required before it can be used as a clinical tool in primary care.
• Limitations the authors state:
  • Asbestos exposure data were missing for most LLPC participants, so the full model could not be used in the cohort. The authors state their results suggest no considerable loss of accuracy from this.
  • The analysis gives only predicted benefits, not actual benefits, and does not recommend a specific risk threshold.
  • Spirometry measurements and genetic markers were not assessed.

Topics

The National Library of Medicine indexes this study under subjects our site covers: Lung cancer.

Medical terms in this study, explained in our Lexicon: Lung Cancer, Cohort Study, Asbestos Exposure, Low-Dose CT Screening.

Related studies

• Impact of low-dose computed tomography screening on lung cancer mortality among asbestos-exposed workers — F. Barbone, F. Barbiero, et al. (2018)
• Development and validation of a lung cancer risk prediction model for African-Americans — C.J. Etzel, S. Kachroo, et al. (2008)
• Pleural plaques in lung cancer screening by low-dose computed tomography: prevalence, association with lung cancer and mortality — M. Silva, N. Sverzellati, et al. (2017)
• Low-dose chest computed tomography screening of subjects exposed to asbestos — K. Kato, K. Gemba, et al. (2018)
• External validation and recalibration of the Brock model to predict probability of cancer in pulmonary nodules using NLST data — A. Winter, D.R. Aberle, et al. (2019)
• Delayed-Phase Enhancement for Evaluation of Malignant Pleural Mesothelioma on Computed Tomography: A Prospective Cohort Study — A. Patel, L. Roshkovan, et al. (2021)

Browse more research in Annals of Internal Medicine, or return to medical research.