Between- and within-reader variability in the assessment of pleural abnormality using the ILO 1980 international classification of pneumoconioses

Our plain-English summary of Between- and within-reader variability in the assessment of pleural abnormality using the ILO 1980 international classification of pneumoconioses, by J. Bourbeau, P. Ernst (American Journal of Industrial Medicine, 1988).

The study

• Authors: J. Bourbeau, P. Ernst
• Journal: American Journal of Industrial Medicine
• Published: 1988;14(5):537-43
• Cited by: 30 articles, as counted by NIH iCite
• Read it: PubMed · Publisher
• In J-M's library: Johns-Manville's company medical library held a copy (J-M no. 000193).

Abstract

Although there are published data concerning reader variability in the assessment of parenchymal pneumoconiotic changes using the ILO Classification of Radiographs, nothing has been published on reader variability with regard to pleural abnormalities. Therefore, in the context of an epidemiologic study, we assessed between- and within-reader variability in the reading of chest radiographs for pleural abnormality using the ILO 1980 International Classification of Radiographs of Pneumoconioses. Chest radiographs of 182 insulation workers interspersed with 24 subjects without documented exposure to asbestos were assembled and read blindly by two readers, reading separately on two occasions, 1 week apart. The results of this study suggest that confident separation of pleural plaques and diffuse pleural thickening may be difficult to achieve using the present guidelines of the ILO 1980 classification. In the evaluation of the width of chest wall pleural abnormality, within-reader agreement improves as the width increases, while between-reader agreement was much less satisfactory. Excellent agreement is obtained in the evaluation of other sites, especially costophrenic angle obliteration and the presence of pleural calcification.

In plain English

This study measured how consistently doctors read chest X-rays for pleural changes (changes in the lining around the lungs) using the ILO 1980 classification. Two chest physicians, both certified B readers, separately and blindly read the films of 182 construction insulators aged 20-50. The films were mixed with those of 24 hospital patients whose medical charts did not mention asbestos exposure. Each physician read every film twice, one week apart. Agreement was measured with the kappa statistic. Both readers found more pleural abnormality in the insulators (42.0% to 53.0% across readers and readings) than in the comparison group (8.0% to 28.0%). When deciding whether a chest-wall change was a pleural plaque or diffuse pleural thickening, reader 1 switched the call 17% of the time between readings. Reader 2 switched it 27% of the time on the right side and 22% on the left. The two readers disagreed with each other 28% and 19% of the time. Agreement on the diaphragm, costophrenic angle and calcification ranged from good to almost perfect. Agreement between readers on the width of chest-wall thickening was generally poor or no better than fair. The authors concluded that confidently separating plaques from diffuse thickening may be difficult under the ILO 1980 criteria. They noted that the comparison group lacked full work histories.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Reader variability (reliability) study of chest radiograph readings, carried out within an epidemiologic study, with agreement measured by the kappa statistic
• Who was studied: Posteroanterior chest radiographs of 182 construction insulators aged 20-50 years, members of the International Association of Heat and Frost Insulators and Asbestos Workers (local 58) who were part of a lung function study. These were interspersed with films of 24 subjects of similar age examined at the Montreal Chest Hospital Center radiology department. Each film was read twice, one week apart, by two chest physicians who were certified B readers.
• Exposure: Work as construction insulators. The article does not give fiber types, products or doses. The comparison group's medical charts were checked for the absence of any mention of prior asbestos exposure or pleural abnormality.
• Main results:
  • Both readers, on both reading occasions, read a higher rate of any pleural abnormality in the exposed insulators (n = 182) than in the nonexposed subjects (n = 24). Exposed group: reader 1 found 43.0% and 53.0%; reader 2 found 48.5% and 42.0%. Nonexposed group: reader 1 found 21.0% and 28.0%; reader 2 found 12.5% and 8.0%.
  • Readers often switched between calling a chest-wall change a pleural plaque and calling it diffuse pleural thickening. Within reader, reader 1 interchanged the type 17% of the time between readings. Reader 2 interchanged it 27% of the time on the right side and 22% on the left side.
  • The two readers disagreed on the type of chest-wall pleural abnormality (plaque or diffuse thickening) 28% of the time on the right side and 19% on the left side.
  • Agreement for diaphragm, costophrenic angle and pleural calcification was excellent to almost perfect within reader and good to excellent between readers. Kappa values: reader 1 had .63, .89 and .64; reader 2 had .83, .70 and .81; between readers the values were .57, .66 and .68.
  • For the width of chest-wall pleural abnormality, within-reader agreement improved as width increased. Between-reader agreement was generally poor or no better than fair, and it did not improve as width increased. Between-reader kappas were .39 for width a, .23 for width b and .00 for width c.
  • Agreement for the extent of chest-wall pleural abnormality was poor to fair, both within and between readers.
  • The prevalence of pleural abnormality in the nonexposed subjects was greater than the authors expected. They suggested this may point to a low threshold for reading pleural abnormality among the readers.
• The authors' conclusion: The authors concluded that confident separation of the two types of chest-wall pleural thickening (pleural plaques and diffuse pleural thickening) may be difficult to achieve using the PA film and the criteria in the ILO 1980 classification.
• Limitations the authors state:
  • There was no comprehensive occupational history for all the nonexposed subjects. Significant transient asbestos exposure, or other previous pleural disease, in that group could not be excluded.
  • Few films showed chest-wall pleural thickening of 5 mm or more, so a small discordance in readings could greatly affect reproducibility at larger widths.
  • Poor between-reader agreement for thin chest-wall abnormality may have resulted from the two readers using different threshold criteria.
  • The readers did not strictly apply the distinction based on costophrenic angle involvement between true diffuse thickening and confluent plaques. The authors state that the expected improvement from using this criterion has not been verified.

Topics

Medical terms in this study, explained in our Lexicon: ILO Classification, Chest X-Ray, Insulator, Pleural Plaques, Diffuse Pleural Thickening, Costophrenic angle, Pleural Calcification.

Related studies

• Variability in interpretation among B-readers in the U.S. Navy Asbestos Medical Surveillance Program — J.G. Muller, J.M. Lieske, et al. (2008)
• Relationships (I) of International Classification of High-resolution Computed Tomography for Occupational and Environmental Respiratory Diseases with the ILO International Classification of Radiographs of Pneumoconioses for parenchymal abnormalities — T. Tamura, N. Suganuma, et al. (2015)
• Inter-reader variability in chest radiography and HRCT for the early detection of asbestos-related lung and pleural abnormalities in a cohort of 636 asbestos-exposed subjects — E. Ochsmann, T. Carl, et al. (2010)
• The relationship between respiratory impairment and asbestos-related pleural abnormality in an active work force — J. Bourbeau, P. Ernst, et al. (1990)
• Development of radiographic abnormality in chrysotile miners and millers — G.M. Eyssen (1980)
• Pathologica, physiological, and radiological correlations in the pneumoconioses — E.A. Gaensler, C.B. Carrington, et al. (1972)

Browse more research in American Journal of Industrial Medicine, or return to medical research.