Overreliance on a single study: there is no real evidence that applying quality criteria to exposure in asbestos epidemiology affects the estimated risk

Our plain-English summary of Overreliance on a single study: there is no real evidence that applying quality criteria to exposure in asbestos epidemiology affects the estimated risk, by D.W. Berman, B.W. Case (The Annals of Occupational Hygiene, 2012).

The study

• Authors: D.W. Berman, B.W. Case
• Journal: The Annals of Occupational Hygiene
• Published: 2012;56(8):869-78
• Cited by: 7 articles, as counted by NIH iCite
• Read it: PubMed · Publisher

Abstract

A critical need exists for reliable risk management policies and practices that can effectively mitigate asbestos-related health threats, and such policies and practices need to be based on sound science that adequately distinguishes hazardous situations from those that are not. Toward that end, the disparate means by which study quality has been addressed in recent meta-analyses used to establish potency factors (K ( L ) and K ( M ) values) for asbestos cancer risks were compared by conducting additional sensitivity analyses. Results suggest that, other than placing undue emphasis on the influence of the K ( L ) and K ( M ) values reported from a single study, there appears to be little to no evidence of a systematic effect of study quality on K ( L ) or K ( M ) values; none of the findings warrant excluding studies from current or future meta-analyses. Thus, we argue that it is better to include as much of the available data as possible in these analyses while formally addressing uncertainty as part of the analysis itself, rather than sequentially excluding studies based on one type of limitation or another. Throwing out data without clearly proving some type of bias is never a good idea because it will limit both the power to test various hypotheses and the confidence that can be placed in any findings that are derived from the resulting, truncated data set. We also believe that it is better to identify the factors that contribute to variation between studies included in a meta-analysis and, by adjusting for such factors as part of a model, showing that the disparate values from individual studies can be reconciled. If such factors are biologically reasonable (based on other evidence) and, if such a model can be shown to fit the data from all studies in the meta-analysis, the model is likely to be predictive of the parameters being evaluated and can then be applied to new (unstudied) environments.

In plain English

This commentary re-examined two meta-analyses (Lenters et al., 2011; Health Council of the Netherlands, 2010) that excluded lower-quality studies. For lung cancer the authors re-analyzed published potency values from 19 studies (17 in a second data set); for mesothelioma they ranked those from 13 studies. The studies covered chrysotile, crocidolite, amosite and mixed exposures. Pooled lung cancer potency in the highest-quality studies was 4.4 or 2.5 times that in all studies, by data set. Without the South Carolina textile cohort, it was 2.4 or 1.0 times. The authors said none appeared significant. Removing South Carolina profoundly affected the results; removing the Libby, Montana vermiculite study did not. For mesothelioma, the two highest-quality studies ranked 5 and 8.5 of 13. Among chrysotile studies, South Carolina ranked 4 of 5. The one higher-ranked New Orleans cement study rested on one mesothelioma case outside the formal cohort. Setting it aside, the authors saw some suggestion of a quality effect but said four studies were too few to test. They concluded that discarding data without proving bias is never a good idea, and invited more rigorous confirmation.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Commentary presenting additional sensitivity analyses (re-analyses) of two published meta-analyses that excluded lower-quality studies
• Who was studied: No new subjects. For lung cancer, the data were published potency (KL) values: 19 studies using the Lenters et al. (2011) values, and 17 studies using the Berman and Crump values. For mesothelioma, the authors ranked 13 of the 14 potency (KM) values reported by Berman and Crump (2008b), without a pooled re-analysis.
• Exposure: Asbestos exposures in the underlying cohorts: chrysotile, crocidolite, amosite and mixed. The cohorts included the South Carolina textile cohort and the Libby, Montana vermiculite miners.
• Main results:
  • Lung cancer: pooled potency for the highest-quality studies compared with all studies gave ratios of 4.4 (Lenters et al. values) and 2.5 (Berman and Crump values). With the South Carolina cohort removed, the ratios were 2.4 and 1.0, the latter meaning no difference. The authors stated that none of the ratios appeared significant.
  • In the trend tests, removing the South Carolina study had a profound effect on the outcome, while removing the Libby study did not. The authors said it was specifically the South Carolina study, rather than high-quality studies in general, that drove the Lenters et al. results.
  • The order in which quality criteria were applied could determine whether a significant trend was observed.
  • For mesothelioma, the authors ranked published study-specific potency values, both across all studies and within fiber-type groups.
  • Mesothelioma, all 13 studies ranked together: the South Carolina textile study ranked 5 of 13 and the Rochdale, England mixed textile study ranked 8.5 of 13. These were the two highest-quality studies, and the middle rank was 7.
  • Mesothelioma, chrysotile studies only: the South Carolina study ranked 4 of 5. The single higher-ranked study, the New Orleans cement factory cohort, rested on one mesothelioma case among individuals not formally included in the cohort.
  • If the New Orleans study was set aside, South Carolina would rank highest of the four remaining chrysotile studies. The authors said this offered some suggestion of a study quality effect, but that the set was too small to test for significance, so the effect could not be concluded to have been demonstrated.
• The authors' conclusion: The authors concluded that, apart from undue emphasis on a single study, there was little to no evidence of a systematic effect of study quality on potency values. They argued that studies should not be excluded from meta-analyses without clear proof of bias.
• Limitations the authors state:
  • The analyses were simplified, and the authors suggested others confirm them with more rigorous sensitivity analyses.
  • There were too few chrysotile mesothelioma studies to test whether the suggested quality effect was significant.
  • The effects of study quality, fiber type and fiber size may be confounded, and the two approaches have not been compared in a common analysis.

Topics

The National Library of Medicine indexes this study under subjects our site covers: Chrysotile.

Medical terms in this study, explained in our Lexicon: Epidemiology, Meta-Analysis.

Related studies

• Applying quality criteria to exposure in asbestos epidemiology increases the estimated risk — A. Burdorf, D. Heederik (2011)
• Quality of evidence must guide risk assessment of asbestos — V. Lenters, A. Burdorf, et al. (2012)
• Quality of evidence must guide risk assessment of asbestos, by Lenters, V; Burdorf, A; Vermeulen, R; Stayner, L; Heederik, D — D.W. Berman, B.W. Case (2013)
• Quality of evidence must guide risk assessment of asbestos, by Lenters, V; Burdorf, A; Vermeulen, R; Stayner, L; Heederik, D — J.T. Hodgson (2013)
• An air quality data analysis system for interrelating effects, standards, and needed source reductions: Part 13--Applying the EPA Proposed Guidelines for Carcinogen Risk Assessment to a set of asbestos lung cancer mortality data — R.I. Larsen (2003)
• I-O Optimise: a novel multinational real-world research platform in thoracic malignancies — S. Ekman, F. Griesinger, et al. (2019)

Browse more research in The Annals of Occupational Hygiene, or return to medical research.