JEMs and incompatible occupational coding systems: effect of manual and automatic recoding of job codes on exposure assignment

Our plain-English summary of JEMs and incompatible occupational coding systems: effect of manual and automatic recoding of job codes on exposure assignment, by T. Koeman, N.S. Offermans, Y. Christopher-de Vries, et al (The Annals of Occupational Hygiene, 2013).

The study

• Authors: T. Koeman, N.S. Offermans, Y. Christopher-de Vries, P. Slottje, P.A. Van Den Brandt, R.A. Goldbohm, H. Kromhout, R. Vermeulen
• Journal: The Annals of Occupational Hygiene
• Published: 2013;57(1):107-14
• Cited by: 27 articles, as counted by NIH iCite
• Read it: PubMed · Publisher

Abstract

Background: In epidemiological studies, occupational exposure estimates are often assigned through linkage of job histories to job-exposure matrices (JEMs). However, available JEMs may have a coding system incompatible with the coding system used to code the job histories, necessitating a translation of the originally assigned job codes. Since manual recoding is usually not feasible in large studies, this is often done by use of automated crosswalks translating job codes from one system to another. We set out to investigate whether automatically translating job codes led to different exposure estimates compared with those resulting from manual recoding using the original job descriptions.

Methods: One hundred job histories were randomly drawn from the Netherlands Cohort Study on diet and cancer (NLCS), using a sampling strategy designed to oversample potentially exposed jobs. This resulted in 220 job codes that were automatically translated from the original Dutch coding system to the International Standard Classification of Occupations (ISCO)-68 and ISCO-88 as well as manually recoded from the job descriptions in the original questionnaire by two coders. Exposure to several agents (i.e. chromium, asbestos, silica, pesticides, aromatic solvents, and extremely low-frequency magnetic fields) was assigned by JEMs based on job codes resulting from automatic and manual recodings.

Results: The agreement between occupational exposure estimates based on the crosswalk versus those based on manual recoding reached a Cohen's Kappa (κ) of 0.66 or higher and were similar to the agreements between the two coders.

Conclusions: Results of this study indicate that using automated crosswalks to recode job codes from one occupational classification system to another results only in a limited loss in agreement in assigned occupational exposure estimates compared with direct manual recoding. Therefore, in this case, crosswalks provide an efficient alternative to the costly and time-consuming direct manual recoding from job history descriptions from questionnaires.

In plain English

This methods study tested whether automatically translating job codes from one classification system to another changes the exposures that job-exposure matrices (tables linking jobs to likely exposures) assign. The authors drew 100 questionnaires from the Netherlands Cohort Study, oversampling blue-collar jobs, which gave 220 jobs. The existing Dutch job codes were translated into the international ISCO-68 and ISCO-88 systems by automatic translation tables called crosswalks. Separately, two experienced coders independently assigned ISCO codes by hand from the original questionnaire job descriptions. Exposures, including asbestos, were rated as not exposed, low or high. Agreement between the two hand coders served as the benchmark. For asbestos, assessed in 210 jobs after 10 that one coder could not code were dropped, agreement (weighted Cohen's Kappa) was 0.72 between the coders, and 0.77 and 0.69 between the crosswalk and each coder. Agreement was lower for some other exposures, notably aromatic solvents. Agreement on the job codes themselves was low. The authors concluded that crosswalk-based exposure estimates performed similarly well to manual recoding. They noted this probably holds as long as the crosswalk translates to a related and less detailed system.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Methods comparison study of exposure assessment: agreement between job-exposure matrix exposure estimates based on automatically translated (crosswalk) job codes versus job codes manually recoded from questionnaire job descriptions by two coders
• Who was studied: 100 questionnaires drawn from the Netherlands Cohort Study (NLCS; 120 852 men and women aged 55–69 years enrolled in 1986), oversampling blue-collar jobs. This gave 220 jobs, of which 210 were analysed for ISCO-68 and 205 for ISCO-88.
• Exposure: Job-exposure matrix estimates (not exposed, low, high) for several agents. Asbestos, silica, chromium, nickel and diesel motor emissions were assigned through DOMJEM to ISCO-68 codes. ALOHA+ and an ELF-MF JEM covered other agents. No asbestos fiber types are specified.
• Main results:
  • Of the 220 jobs, 10 were not codeable according to one coder and 5 more were excluded from the ISCO-88 analyses. This left 210 jobs for the ISCO-68 analysis and 205 for the ISCO-88 analysis.
  • Asbestos (assigned through DOMJEM to ISCO-68 codes, 210 jobs): weighted Cohen's Kappa was 0.72 between recoders 1 and 2 (the reference), 0.77 between recoder 1 and the CBS-7 crosswalk, and 0.69 between recoder 2 and the CBS-7 crosswalk.
  • Asbestos prevalence among 210 jobs: not exposed 176 (84%) by recoder 1, 179 (85%) by recoder 2 and 171 (81%) by the CBS-7 crosswalk. Low exposed: 31, 28 and 36. High exposed: 3 by each method.
  • For the DOMJEM exposures, agreement between the CBS-7–ISCO-68 crosswalk and either recoder was generally equal to or higher than agreement between the two recoders.
  • For aromatic solvents, agreement between the CBS-7–ISCO-88 crosswalk and recoders 1 and 2 was 0.69 and 0.66, compared with 0.78 between the two recoders.
  • For aromatic solvents, agreement between the CBS-4–ISCO-88 crosswalk and recoders 1 and 2 was lower still, at 0.53 and 0.55.
  • Agreement on the job codes themselves was low. For ISCO-68 at the occupational group level it was 36% between the recoders and at most 43% between a recoder and the crosswalk.
  • When the analysis was restricted to jobs on which both recoders agreed on the code, Kappas between crosswalks and recoders mostly exceeded 0.75. The exception was nickel, at 0.66.
• The authors' conclusion: The authors concluded that exposure estimates obtained through crosswalk translation of job codes performed similarly well to direct manual recoding. They added that this probably holds as long as the crosswalk translates to a related and less detailed coding system.
• Limitations the authors state:
  • There was no gold standard for the job codes or the exposure estimates. Agreement between the two recoders was used as the benchmark instead.
  • Crosswalk performance also depended on the original coder who assigned the Dutch CBS-7 codes. The authors argued this would more likely make their results an underestimate of crosswalk performance.
  • The authors said the finding probably holds as long as the crosswalk translates to a related and less detailed coding system.

Topics

Medical terms in this study, explained in our Lexicon: Occupational Exposure, Cohort Study.

Related studies

• Comparison of expert and job-exposure matrix-based retrospective exposure assessment of occupational carcinogens in The Netherlands Cohort Study — N.S. Offermans, R. Vermeulen, et al. (2012)
• Occupational asbestos exposure and risk of pleural mesothelioma, lung cancer, and laryngeal cancer in the prospective Netherlands cohort study — N.S. Offermans, R. Vermeulen, et al. (2014)
• Microdetermination of carbon and hydrogen Automatic combustion in a dynamic system — N. Gel'man (1969)
• Lung Cancer Risks Associated with Occupational Exposure to Pairs of Five Lung Carcinogens: Results from a Pooled Analysis of Case-Control Studies (SYNERGY) — A. Olsson, L. Bouaoun, et al. (2024)
• Hierarchical regression for multiple comparisons in a case-control study of occupational risks for lung cancer — M. Corbin, L. Richiardi, et al. (2012)
• Parental occupational exposure and risk of childhood central nervous system tumors: a pooled analysis of case-control studies from Germany, France, and the UK — C. Huoi, A. Olsson, et al. (2014)

Browse more research in The Annals of Occupational Hygiene, or return to medical research.