An assessment of the Environmental Protection Agency's asbestos hazard evaluation algorithm

Our plain-English summary of An assessment of the Environmental Protection Agency's asbestos hazard evaluation algorithm, by M.E. Findley, V.E. Rose, G.R. Cutter, et al (American Journal of Public Health, 1983).

The study

• Authors: M.E. Findley, V.E. Rose, G.R. Cutter, R.A. Windsor
• Journal: American Journal of Public Health
• Published: 1983;73(10):1179-81
• Cited by: 6 articles, as counted by NIH iCite
• Read it: PubMed · Publisher · Free full text
• In J-M's library: Johns-Manville's company medical library held a copy (J-M no. 004639).

Abstract

We performed an exploratory evaluation of the Environmental Protection Agency's algorithm for assessment of risk of exposure to asbestos in buildings. Observers scored five sites using the algorithm. One group (N = 23) conducted their assessments relying only on written instructions, while another (N = 20) was trained in the use of the algorithm. Five professional industrial hygienists, experienced in risk assessment, also used the algorithm on the same sites. A wide range of highly variable results, for both untrained and trained observers coupled with poor comparability with expert's scores raise concern as to the reliability of this assessment tool. The development of a valid and reliable "lay" index of risk is still needed.

In plain English

This exploratory study tested a draft algorithm proposed by the US Environmental Protection Agency for judging the risk of asbestos exposure in buildings. The algorithm rates factors such as material condition, water damage and friability, combines them into one number, and reads it on a scale running from deferring action to removal. Forty-three graduate students (19 in educational administration, 24 in public health) scored five sites with sprayed materials, including laboratories, classrooms, hallways and a parking garage. Twenty-three used only written instructions. The other 20 had two hours of training first. Five experienced industrial hygienists also scored the sites and then reached consensus scores. Training made no significant difference to average scores. At four of the five sites, individual scores fell in all three action ranges. Of the students' recommendations, 56 per cent matched the experts' consensus, 35 per cent called for stricter action and 9 per cent for less strict action. The experts' own scores also differed. The authors concluded that the algorithm, in its present form, did not appear to be a reliable measure of potential asbestos hazard in buildings.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Exploratory evaluation of the reliability of a risk-scoring tool. Observers were randomly split into an untrained ("standard") group and a trained ("enriched") group, and their scores were compared with those of an expert panel.
• Who was studied: The observers were 19 graduate students in educational administration and 24 graduate public health students (43 in total), randomly divided into a standard group (N=23) and an enriched group (N=20). A panel of 5 experienced industrial hygienists served as the expert group. All of them scored five building sites.
• Exposure: Sprayed asbestos-containing materials at five sites (laboratories, classrooms, hallways and a parking garage). The sites varied in degree of damage and included both enclosed and exposed surfaces. The materials were assessed with the EPA's draft Asbestos Exposure Algorithm. No fiber types are named.
• Main results:
  • There were no significant differences in mean scores between the standard (untrained) and enriched (trained) groups, or by background (educational administration versus public health).
  • The enriched group computed 72 per cent of its scores correctly, against 54 per cent for the standard group. The authors described this as an error rate of 2 per cent versus 4 per cent and stated that the difference was not significant.
  • At four of the five sites, at least one observer's score fell in each of the three corrective action ranges, from no action to removal.
  • At site 4 (a parking garage), which the authors reported as the most variable, there were 21 scores in the deferred or no action range, 15 in the encapsulation/enclosure range and 7 in the removal range.
  • In reaching consensus, all five experts agreed on 29 of the 35 possible factor ratings. They had the most difficulty agreeing on material condition and water damage. The authors also reported important differences in scores among the professionals.
  • Compared with the experts' consensus, 56 per cent of observers' recommendations agreed, 35 per cent called for more stringent action, and 9 per cent called for less stringent action.
  • Treated as a screening tool for whether air sampling was needed, judged against the experts' consensus, the algorithm scores had an average sensitivity of 98 per cent and a specificity of 53 per cent. The authors called this specificity unacceptably low.
• The authors' conclusion: The authors concluded that, based on this limited study, the EPA algorithm in its present form did not appear to be a reliable measure of potential hazardous asbestos exposure in buildings. They recommended further study of its reliability before it is included in any regulation.
• Limitations the authors state:
  • The authors described the study as limited and exploratory.
  • The test sites were not fully equivalent to situations commonly found in elementary and secondary schools.
  • The authors did not know which sites should actually have been air sampled, so the sensitivity and specificity figures rested on the assumption that the experts' consensus was correct.
  • The training given was brief. The authors stated that future training would need to be more powerful and extensive.

Related studies

• EPA as a public health agency — B.D. Goldstein (1988)
• Comparing epidemiologic studies of ingested asbestos for use in risk assessment — L.S. Erdreich (1983)
• Asbestos in drinking water: a status report — J.A. Cotruvo (1983)
• EPA and asbestos removal — J. Berry (1990)
• How do you interpret regulations: through science or agency rules? — J.H. Lange (2002)
• Asbestos removal, health hazards, and the EPA. Council on Scientific Affairs, American Medical Association — (1991)

Browse more research in American Journal of Public Health, or return to medical research.